Tag Archives: scientists

Scientists have difficulty explaining p-values

Posted on by

Scientists regularly use p-values to evaluate their findings but apparently have difficulty explain exactly what they mean:

To be clear, everyone I spoke with at METRICS could tell me the technical definition of a p-value — the probability of getting results at least as extreme as the ones you observed, given that the null hypothesis is correct — but almost no one could translate that into something easy to understand.

It’s not their fault, said Steven Goodman, co-director of METRICS. Even after spending his “entire career” thinking about p-values, he said he could tell me the definition, “but I cannot tell you what it means, and almost nobody can.” Scientists regularly get it wrong, and so do most textbooks, he said. When Goodman speaks to large audiences of scientists, he often presents correct and incorrect definitions of the p-value, and they “very confidently” raise their hand for the wrong answer. “Almost all of them think it gives some direct information about how likely they are to be wrong, and that’s definitely not what a p-value does,” Goodman said.

We want to know if results are right, but a p-value doesn’t measure that. It can’t tell you the magnitude of an effect, the strength of the evidence or the probability that the finding was the result of chance.

So what information can you glean from a p-value? The most straightforward explanation I found came from Stuart Buck, vice president of research integrity at the Laura and John Arnold Foundation. Imagine, he said, that you have a coin that you suspect is weighted toward heads. (Your null hypothesis is then that the coin is fair.) You flip it 100 times and get more heads than tails. The p-value won’t tell you whether the coin is fair, but it will tell you the probability that you’d get at least as many heads as you did if the coin was fair. That’s it — nothing more. And that’s about as simple as I can make it, which means I’ve probably oversimplified it and will soon receive exasperated messages from statisticians telling me so.

Complicated but necessary? This can lead to fun situations when teaching statistics: students need to know enough to do the statistical work and evaluate findings (we at least need to know what to do with a calculated p-value, even if we don’t quite understand what it means) but explaining the complexity of some of these techniques wouldn’t necessarily help the learning process. In fact, the more you learn about statistics, you tend to find that the various methods and techniques have limitations even as they can help us better understand phenomena.

Recent sociological findings: many evangelicals think science and religion can work together, few highly invested in evolution/creation debate

Posted on by

Two recent studies suggest there may be less conflict between religious Americans and science than is typically portrayed.

1. Sociologist Elaine Ecklund on how religion and science interact:

“We found that nearly 50 percent of evangelicals believe that science and religion can work together and support one another,” Ecklund said. “That’s in contrast to the fact that only 38 percent of Americans feel that science and religion can work in collaboration.”…

  • Nearly 60 percent of evangelical Protestants and 38 percent of all surveyed believe “scientists should be open to considering miracles in their theories or explanations.”
  • 27 percent of Americans feel that science and religion are in conflict.
  • Of those who feel science and religion are in conflict, 52 percent sided with religion.
  • 48 percent of evangelicals believe that science and religion can work in collaboration.
  • 22 percent of scientists think most religious people are hostile to science.
  • Nearly 20 percent of the general population think religious people are hostile to science.
  • Nearly 22 percent of the general population think scientists are hostile to religion.
  • Nearly 36 percent of scientists have no doubt about God’s existence.

RUS is the largest study of American views on religion and science. It includes the nationally representative survey of more than 10,000 Americans, more than 300 in-depth interviews with Christians, Jews and Muslims — more than 140 of whom are evangelicals — and extensive observations of religious centers in Houston and Chicago.

Ecklund comes to similar conclusions in her 2010 book about scientists and religious faith Science vs. Religion.

2. Sociologist Jon Hill on how Americans view the evolution debate:

As part of a recent project funded by the BioLogos Foundation, I have fielded a new, nationally representative survey of the American public: The National Study of Religion and Human Origins (NSRHO).

Unlike existing surveys, this one includes extensive questions about human origins that allow us to develop a more accurate portrait of what the general public—and, in particular, Christians—actually believe. The survey includes questions on belief in human evolution, divine involvement, the existence of Adam and Eve, historical timeframe, original sin, and more. For each of these questions, participants are allowed to respond with “not at all sure” about what they believe. If they claim a position, they are also asked to rate how confident they are that their belief is correct. Lastly, they are asked to report how important having the right beliefs about human origins is to them personally…

If only eight percent of respondents are classified as convinced creationists whose beliefs are dear to them, and if only four percent are classified as atheistic evolutionists whose beliefs are dear to them, then perhaps Americans are not as deeply divided over human origins as polls have indicated. In fact, most Americans fall somewhere in the middle, holding their beliefs with varying levels of certainty. Most Americans do not fall neatly into any of the existing camps, and only a quarter claimed their beliefs were important to them personally.

So what does this mean for the church? I think it shows that most people, even regular church-going evangelicals, are not deeply entrenched on one side of a supposed two-sided battle. Certainly, the issue divides Christians. But Christian beliefs about human origins are complex. There’s no major single chasm after all.

In other words, the average religious American doesn’t have think this issue is a matter of life and death, even if the rhetoric from both sides is that the other is a clear enemy.

Interpreting data regarding scientists and religion

Posted on by

In looking at some data regarding what scientists think about religion, a commentator offers this regarding interpreting sociological data:

The point about asking such questions is not because we know the answers but to emphasise that the interpretation of sociological data is a tricky business. From the perspective of science, ants and humans are far more complex than stars and rocks. A discussion of atheism and science in the US context leads us straight to a discussion of the structure of the American educational system, the role of elites, the present polarisation of the political electorate along religious faultlines, and much else besides…

The challenge then is to think hard about the complex data and not be too dogmatic about the interpretations.

When the phrase “tricky business” is used, it sounds like it is referring to the complex nature of the social world. In order to understand the relationship between science and religion, one must account for a variety of possible factors. It is one thing to say that there are multiple possible interpretations of the same data, another to say that some twist data to support their personal interpretations, and another to suggest that we can get to a correct or right interpretation if we properly account for complexity.

While this commentary is ultimately about using caution when interpreting statistics regarding the religious beliefs of scientists, it also is a little summary of social science research regarding the religious beliefs of scientists. The 2010 study Science vs. Religion is discussed as well as a few other works.

Risk of California superstorm – and what should be done about it?

Posted on by

Human beings have a remarkable capacity to build settlements in harsh conditions. Recently, I have wondered what would possess settlers in the 1800s to live in the Upper Midwest with its harsh winters. A classic example of a place with both advantages and disadvantages: California. On one hand, a temperate to warm climate with a wonderful range of habitats (mountains to coast) and rich farmland in the middle of the state.

And yet, California has a number of natural threats. The latest: scientists predicting a superstorm that could flood the state for an extended period.

A group of more than 100 scientists and experts say in a new report that California faces the risk of a massive “superstorm” that could flood a quarter of the state’s homes and cause $300 billion to $400 billion in damage. Researchers point out that the potential scale of destruction in this storm scenario is four or five times the amount of damage that could be wrought by a major earthquake…

The threat of a cataclysmic California storm has been dormant for the past 150 years. Geological Survey director Marcia K. McNutt told the New York Times that a 300-mile stretch of the Central Valley was inundated from 1861-62. The floods were so bad that the state capital had to be moved to San Francisco, and Governor Leland Stanford had to take a rowboat to his own inauguration, the report notes. Even larger storms happened in past centuries, over the dates 212, 440, 603, 1029, 1418, and 1605, according to geological evidence…

The scientists built a model that showed a storm could last for more than 40 days and dump 10 feet of water on the state. The storm would be goaded on by an “atmospheric river” that would move water “at the same rate as 50 Mississippis discharging water into the Gulf of Mexico,” according to the AP. Winds could reach 125 miles per hour, and landslides could compound the damage, the report notes.

Such a superstorm is hypothetical but not improbable, climate researchers warn. “We think this event happens once every 100 or 200 years or so, which puts it in the same category as our big San Andreas earthquakes,” Geological Survey scientist Lucy Jones said in a press release.

If this is a real possibility, the question then becomes what the state should do about it. It is another example of weighting risks: should the state implement all sorts of rules and plans to limit the possible damage or should they simply go on with life and deal with the consequences when they come? Of course, California isn’t the only place that faces such questions: hurricanes pose a similar threat on the East or Gulf Coasts and many communities have homes or businesses built on flood plains.

Regardless of what California does with this information, perhaps this can become additional fodder for disaster movies. I can see the plot line now: California is hit with a major storm followed by a major earthquake with both accompanied with major mudslides followed by our set of heroes running for the hills…you’ve seen this plot line before. But this flood of 1861-1862 does sound intriguing – perhaps more information about this past event would help current officials plan for future events.