Tag Archives: science

Quick Review: the Sherlock Holmes stories

Posted on by

One of my reading projects this summer was to read all of the Sherlock Holmes short stories (56) and novels (4). I enjoyed reading these classics and here are a few thoughts about the well-known detective and his sidekick Watson:

1. I don’t read a lot of mysteries but I can see that more recent detectives (books, TV, movies) have hints of Holmes. Holmes is the classic scientific detective, reasoning his way through tough cases. There has to be a line from Holmes to Hercule Poirot to Adrian Monk. Of course, Holmes’ emphasis on science also emerges as the larger society moves more toward a belief of science and progress.

2. I’m not sure that I like Sherlock Holmes in the end and I’m not sure Doyle wanted people to like him but rather wanted people to be impressed by him. Holmes certainly has a sharp mind but he is given to mood swings, using opium, and rarely shows a non-scientific side. For example, there are a few points in the later stories where Watson seems thrilled that Holmes reveals some warm feelings for his companion. Holmes is a sort of modern renaissance man but is a limited person.

3. Even with the presence of Professor Moriarty, there was one big difference with recent stories: there is a lack of a major villain. Indeed, Holmes does a lot of one-off cases and there are a few recurring characters.

4. After reading all of these stories, I’m not sure I could remember the details of many of them. I liked the four novels the most as there was room to develop the cases and have more twists and turns.

5. I had the opportunity to read most of these stories in the Oxford annotated editions (see an example here). At first, I thought this would be a hindrance (that long introduction, the extensive footnotes) but I really grew to enjoy this. This particularly came in handy with the novels The Gang of Four and A Study in Scarlet as the footnotes described how Doyle built the stories around interesting true events. I didn’t read all of the footnotes (and they truly seemed to be extensive – and occasionally esoteric) but the introductions were helpful.

6. I wish I had read these all in chronological order.

7. I suspect it would have been very different to read these all in the serial form in which they were released.

Sharing data among scientists vs. “Big Data”

Posted on by

In a quest to make data available to other researchers to verify research results, researchers have come up against one kind of data that is not made publicly available: “big data” from big Internet firms.

The issue came to a boil last month at a scientific conference in Lyon, France, when three scientists from Google and the University of Cambridge declined to release data they had compiled for a paper on the popularity of YouTube videos in different countries.

The chairman of the conference panel — Bernardo A. Huberman, a physicist who directs the social computing group at HP Labs here — responded angrily. In the future, he said, the conference should not accept papers from authors who did not make their data public. He was greeted by applause from the audience…

At leading social science journals, there are few clear guidelines on data sharing. “The American Journal of Sociology does not at present have a formal position on proprietary data,” its editor, Andrew Abbott, a sociologist at the University of Chicago, wrote in an e-mail. “Nor does it at present have formal policies enforcing the sharing of data.”

The problem is not limited to the social sciences. A recent review found that 44 of 50 leading scientific journals instructed their authors on sharing data but that fewer than 30 percent of the papers they published fully adhered to the instructions. A 2008 review of sharing requirements for genetics data found that 40 of 70 journals surveyed had policies, and that 17 of those were “weak.”

Who will win the battle between proprietary data and science? The article makes it sound like scientists are all on one side, particularly because of an interest in fighting issues like scientific fraud. At the same time, scientific journals don’t seem to be “enforcing” their guidelines or the individual scientists who are publishing in these journals aren’t following these guidelines.

The other side of this debate is not presented in this story: what do these big Internet firms, like Google, Yahoo, and Facebook think about sharing this data? This is not a small issue: these firms are spending a good amount of money on analyzing this data and probably hoping to use it for their own business and research purposes. For example, Microsoft recently set up a lab with several well-known researchers in New York City. Would the social scientists who work in such labs want to insist that the data be open? Should these companies have to open up their proprietary data to satisfy the requirements of the larger scientific community?

I suspect this will be an ongoing issue as social scientists look to analyze more innovative data that big companies have collected and that are more difficult for researchers to collect on their own. Will researchers be willing to forgo sharing this kind of data with the wider scientific community if they can get their hands on unique data?

Increase in retractions of scientific articles tied to problems in scientific process

Posted on by

Several scientists are calling for changes in how scientific work is conducted and published because of a rise in retracted articles:

Dr. Fang became curious how far the rot extended. To find out, he teamed up with a fellow editor at the journal, Dr. Arturo Casadevall of the Albert Einstein College of Medicine in New York. And before long they reached a troubling conclusion: not only that retractions were rising at an alarming rate, but that retractions were just a manifestation of a much more profound problem — “a symptom of a dysfunctional scientific climate,” as Dr. Fang put it.

Dr. Casadevall, now editor in chief of the journal mBio, said he feared that science had turned into a winner-take-all game with perverse incentives that lead scientists to cut corners and, in some cases, commit acts of misconduct…

Last month, in a pair of editorials in Infection and Immunity, the two editors issued a plea for fundamental reforms. They also presented their concerns at the March 27 meeting of the National Academies of Sciences committee on science, technology and the law.

Here is what Fang and Casadevall suggest may help reduce these issues:

To change the system, Dr. Fang and Dr. Casadevall say, start by giving graduate students a better understanding of science’s ground rules — what Dr. Casadevall calls “the science of how you know what you know.”

They would also move away from the winner-take-all system, in which grants are concentrated among a small fraction of scientists. One way to do that may be to put a cap on the grants any one lab can receive.

In other words, give graduate students more training in ethics and the sociology of science while also redistributing scientific research money so that more researchers can be involved. There is a lot to consider here. Of course, there might always be researchers tempted to commit fraud yet these scientists are arguing that the current system and circumstances needs to be tweaked to fight this. Graduate students and young faculty are well aware of what they have to do: publish research in the highest-ranked journals they can. Jobs and livelihoods are on the line. With that pressure, it makes sense that some may resort to unethical measures to get published.

Three other thoughts:

1. How often is social science research retracted? If it is infrequent, should it happen more often?

2. Even if an article or study is retracted, this doesn’t solve the whole issue as that work may have been cited a lot and become well known. Perhaps the bigger problem is “erasing” this study from the collective science memory. This reminds me of newspaper corrections; when you go find the original printing, you don’t know there was a later correction. The same thing can happen here: scientific studies can have long lives.

3. Should disciplines or journals have groups that routinely assess the validity of research studies? This would go beyond peer review and give a group the authority to ask questions about suspicious papers. Alas, this still wouldn’t catch even most of the problematic papers…

Quick Review: The Immortal Life of Henrietta Lack

Posted on by

After a few people mentioned a particular New York Times bestseller to me recently, I decided to read The Immortal Life of Henrietta Lack. While the story itself was interesting, there is a lot of material here that could be used in research methods and ethics classes. A few thoughts about the book:

1. The story is split into two narratives. One is about both the progress science has made with a Lack’s cells but also the struggle of her family to understand what actually has been done with her cells. The story of scientific progress is unmistakable: we have come a long way in identifying and curing some diseases in the last sixty years. (This narrative reminded me of the book The Emperor of All Maladies.)

2. The second narrative is about the personal side of scientific research and how patients and relatives interpret what is going on. The author initially finds that the Lacks know very little about how their sister or mother’s cells have been used. These problems are compounded by race, class, and educational differences between the Lacks and the doctors utilizing Henrietta’s cells. In my opinion, this aspect is understated in this book. At the least, this is a reminder about how inequality can affect health care. But I think this personal narrative is the best part of the book. When I talk in class about the reasons for Institutional Review Boards, informed consent, and ethics, students often wonder how much social science research can really harm people. As this book discusses, there are some moments in relatively recent history that we would agree were atrocious: Nazi experiments, the Tuskegee experiments, experiments in Guatemala, and so on. Going beyond those egregious cases, this book illustrates the kind of mental and social harm that can result from research even if using Henrietta’s cells never physically harmed the Lacks. I’m thinking about using some sections of this narrative in class to illustrate what could happen; even if new research appears to be safe, we have to make sure we are protecting our research subjects.

3. This book reminded me of the occasional paternalistic side of the medical field. This book seems to suggest this isn’t just an artifact of the 1950s or a racial division; doctors appear slow in addressing concerns some people might have about the use of human tissue in research. I realize that there is a lot at stake here: the afterward of the book makes clear how difficult it would be to regulate this all and how this might severely limit needed medical research. At the same time, doctors and other medical professionals could go further in explaining the processes and the possible outcomes to patients. Perhaps this is why the MCAT is moving toward involving more sociology and psychology.

4. There is room here to contrast the discussions about using body tissue for research and online privacy. In both cases, a person is giving up something personal. Are people more disturbed by their tissue being used or their personal information being used and sold online?

All in all, this book discusses both scientific breakthroughs, how patients can be hurt by the system, and a number of ethical issues that have yet to be resolved.

“Scientists and scientific studies have a minimal effect on public opinion” about global warming

Posted on by

While one might think that scientific data and reports are convincing, a sociologist argues that these matter little in the debate over global warming:

“Scientists and scientific studies have a minimal effect on public opinion,” says Drexel University sociologist Robert Brulle, lead author of a new climate attitude study in the Climatic Change journal. “What really drives public opinion on climate change are the ways that political elites describe the science.”…

In the current Climatic Change journal, Brulle and colleagues looked at 74 public opinion surveys from 2002 to 2010, in a bid to figure out the contradiction in opinions between experts and everyone else…

“The science doesn’t matter because the science isn’t the real issue,” Brulle adds. “It’s about politics and money.” All we have with climate change, he suggests, is politicians taking sides in an economic debate over whether we should spend money to address climate change, or not (with one side very strongly opposed), and hiding behind a smokescreen of debate about settled science to avoid making those issues clear.

Brulle is suggesting that instead of debating how much we should respond to global warming (which seems like an interesting debate to have in itself), the debate has turned to the credibility of the actual science. So if conservatives admit that there is warming, then they would have to admit that money needs to be spent on fighting it and they don’t want to do that? There seem to be two issues here: the actual data and then the value judgments about what should be done.

I’ve been seeing reports on Brulle’s findings for several months now. If he is correct, are politicians taking notes about how to change public debates? At the same time, I imagine it is more difficult to make the case for spending money on environmental concerns with such economic issues (see the Keystone pipeline debate).

I wonder if there are other areas where there is something similar going on and scientific studies have little impact. If there is a common view that science is the province of liberals and elitists, how many people will trust what it has to say?

Journal editors push authors to add citations to improve impact factors?

Posted on by

A new study in Science suggests that some journal editors push authors to include citations in their soon-t0-be published studies to boost the reputation of their journals:

A system of “impact factors”, tied to references listed in studies, pervades the scholarly enterprise, notes survey author Allen Wilhite and Eric Fong of the University of Alabama in Huntsville, who reports the survey of 6,672 researchers in economics, sociology, psychology, and business research in the current Science journal. The survey covered journal editor behavior from 832 publications.

Overall, about 20% of survey respondents say that a journal editor had coerced extra citations to their own journal from them. Broadly, journals with higher impact factors attract more prestige, advertising and power in hiring and firing decisions in scholarly circles, the authors note, giving journal editors an incentive to extort added citations to their publications in the studies they consider for publication. “(T)he message is clear: Add citations or risk rejection,” write the authors.

In particular, younger professors with few co-authors who need publications to keep their jobs reported the most pressure. Business journal editors coerced the most often, followed by economics, and then psychology and other social sciences. As well, “journals published by commercial, for-profit companies show significantly greater use of coercive tactics than journals from university presses. Academic societies also coerce more than university presses.”

Less than 7% of the respondents thought researchers would resist this coercion, so desperate for publication are professors. “Although this behavior is less common in economics, psychology, and sociology, these disciplines are not immune—every discipline reported multiple instances of coercion. And there are published references to coercion in fields beyond the social sciences,” concludes the survey report…

While I’m glad to see that sociology seems to be toward the bottom of this list, this is still a problem. In some ways, this is not surprising as many in academia feel the pressure to make their work stand out.

However, I think you could ask broader questions about the system of citations. Here are a few other ideas:

1. Do researchers feel pressure to add citations to articles simply to reach a “magic number” or to have enough so that it looks like they have “properly” scoped out the topic?

2. How much have citations increased with the widespread use of online databases that make it much easier to find articles?

2a. Since I assume this has increased the number of citations, does this lead to “better research”?

3. When choosing what articles to cite, how much are researchers influenced by how many other people have cited the article (supposedly a measure of its value) and the impact factors of the journal the article is in?

Thinking about Weber as climate change may be the latest issue to join the culture wars

Posted on by

Michael Gerson discusses why climate change has become one of the hot-button issues in the culture wars:

What explains the recent, bench-clearing climate brawl? A scientific debate has been sucked into a broader national argument about the role of government. Many political liberals have seized on climate disruption as an excuse for policies they supported long before climate science became compelling — greater federal regulation and mandated lifestyle changes. Conservatives have also tended to equate climate science with liberal policies and therefore reject both.

The result is a contest of questioned motives. In the conservative view, the real liberal goal is to undermine free markets and national sovereignty (through international environmental agreements). In the liberal view, the real conservative goal is to conduct a war on science and defend fossil fuel interests. On the margin of each movement, the critique is accurate, supplying partisans with plenty of ammunition.

No cause has been more effectively sabotaged by its political advocates. Climate scientists, in my experience, are generally careful, well-intentioned and confused to be at the center of a global controversy. Investigations of hacked e-mails have revealed evidence of frustration — and perhaps of fudging but not of fraud. It is their political defenders who often discredit their work through hyperbole and arrogance. As environmental writer Michael Shellenberger points out, “The rise in the number of Americans telling pollsters that news of global warming was being exaggerated began virtually concurrently with the release of Al Gore’s movie, ‘An Inconvenient Truth.’”

The resistance of many conservatives to arguments about climate disruption is magnified by class and religion. Tea Party types are predisposed to question self-important elites. Evangelicals have long been suspicious of secular science, which has traditionally been suspicious of religious influence. Among some groups, skepticism about global warming has become a symbol of social identity — the cultural equivalent of a gun rack or an ichthus.

If Gerson is correct, the battle over climate change is simply a proxy battle. In fact, then we could probably assume that other issues will come along that will also become part of the culture wars. The fervor over the climate change issue will lessen at some point and another concern will become a flashpoint.

All of this seems related to what I had one of my classes recently read: Weber’s take on “value-free” or “value-neutral” sociology. This could help explain a few things:

1. Distrust of elites, particularly academics, is part of the issue. One way to fight elites/academics is to simply suggest that they are biased. Weber suggests all scientists have some biases. However, there are ways to do science, such as subjecting your work to others with a scientific mindset, to minimize these biases. As I recently argued, just because one scientist may have committed fraud or because some scientists have clear aims does not mean that all science is suspect.

2. Weber suggests that scientists need to be clear when they are speaking as scientists looking at facts and individuals proposing courses of action. Mixing facts and ideals or policies can lead to issues. In this particular situation, I would guess conservatives think the scientists are not just exploring the scientific facts but are also pushing “an agenda.” Indeed, Gerson ends this piece by suggesting we need to put “some distance between science and ideology.” Of course, plenty of scientists are religious but the (perceived) mixing of facts and goals can be problematic.

3. In writing his piece, Weber was trying to set guidelines for a journal where people of a scientific mindset could debate sociology and facts. It is interesting that Gerson notes that opposition to secular science is now part of the subcultural identity of some religious groups, making it more difficult to have conversations because attacking/defending one’s identity is contentious. If one doesn’t want to debate facts, how can one have a conversation about science?

Don’t dismiss social science research just because of one fradulent scientist

Posted on by

Andrew Ferguson argued in early December that journalists fall too easily for bad academic research. However, he seems to base much of his argument on the actions of one fraudulent scientist:

Lots of cultural writing these days, in books and magazines and newspapers, relies on the so-called Chump Effect. The Effect is defined by its discoverer, me, as the eagerness of laymen and journalists to swallow whole the claims made by social scientists. Entire journalistic enterprises, whole books from cover to cover, would simply collapse into dust if even a smidgen of skepticism were summoned whenever we read that “scientists say” or “a new study finds” or “research shows” or “data suggest.” Most such claims of social science, we would soon find, fall into one of three categories: the trivial, the dubious, or the flatly untrue.

A rather extreme example of this third option emerged last month when an internationally renowned social psychologist, Diederik Stapel of Tilburg University in the Netherlands, was proved to be a fraud. No jokes, please: This social psychologist is a fraud in the literal, perhaps criminal, and not merely figurative, sense. An investigative committee concluded that Stapel had falsified data in at least “several dozen” of the nearly 150 papers he had published in his extremely prolific career…

But it hardly seems to matter, does it? The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior…

Who cares? The experiments are preposterous. You’d have to be a highly trained social psychologist, or a journalist, to think otherwise. Just for starters, the experiments can never be repeated or their results tested under controlled conditions. The influence of a hundred different variables is impossible to record. The first group of passengers may have little in common with the second group. The groups were too small to yield statistically significant results. The questionnaire is hopelessly imprecise, and so are the measures of racism and homophobia. The notions of “disorder” and “stereotype” are arbitrary—and so on and so on.

Yet the allure of “science” is too strong for our journalists to resist: all those numbers, those equations, those fancy names (say it twice: the Self-Activation Effect), all those experts with Ph.D.’s!

I was afraid that the actions of one scientist might taint the work of many others.

But there are a couple of issues here and several are worth pursuing:

1. The fact that Stapel committed fraud doesn’t mean that all scientists do bad work. Ferguson seems to want to blame other scientists for not knowing Stapel was committing fraud – how exactly would they have known?

2. Ferguson doesn’t seem to like social psychology. He does point to some valid methodological concerns: many studies involve small groups of undergraduates. Drawing large conclusions from these studies is difficult and indeed, perhaps dangerous. But this isn’t all social psychology is about.

2a. More generally, Ferguson could be writing about a lot of disciplines. Medical research tends to start with small groups and then decisions are made. Lots of research, particularly in the social sciences, could be invalidated if Ferguson was completely right. Ferguson really would suggest “Most such claims of social science…fall into one of three categories: the trivial, the dubious, or the flatly untrue.”?

3. I’ve said it before and I’ll say it again: journalists need more training in order to understand what scientific studies mean. Science doesn’t work in the way that journalists suggests where there is a steady stream of big findings. Rather, scientists find something and then others try to replicate the findings in different settings with different populations. Science is more like an accumulation of evidence than a lot of sudden lightning strikes of new facts. One small study of undergraduates may not tell us much but dozens of such experiments among different groups might.

4. I can’t help but wonder if there is a political slant to this: what if scientists were reporting positive things about conservative viewpoints? Ferguson complains that measuring things like racism and homophobia are difficult but this is the nature of studying humans and society. Ferguson just wants to say that it is all “arbitrary” – this is simply throwing up our hands and saying the world is too difficult to comprehend so we might as well quit. If there isn’t a political edge here, perhaps Ferguson is simply anti-science? What science does Ferguson suggest is credible and valid?

In the end, you can’t dismiss all of social psychology because of the actions of one scientist or because journalists are ill-prepared to report on scientific findings.

h/t Instapundit

Untangling the effects of TV watching on mortality

Posted on by

Interpreting the results of studies can be difficult, particularly if one confuses a correlation (indicating some relationship between two variables) and a direct causal relationship (where one variable causes another). This usually is translated into the common phrase “correlation, not causation” which is illustrated in this example from Entertainment Weekly:

Researchers in Australia are reporting that, on average, every hour spent watching television after the age of 25 decreases the amount you live by 22 minutes.

“As a rule, the more time we spend watching TV, the more time we spend eating mindlessly in front of the TV, and the less time we spend being physically active,” explained Dr. David L. Katz, director of the Prevention Research Center at Yale University School of Medicine to HealthDay.com. “More eating and less physical activity, in turn, mean greater risk for obesity, and the chronic diseases it tends to anticipate, notably diabetes, heart disease and cancer.”

Before you throw your soul-sucking flat screen out the window, here’s a key thing to remember:

TVs are not like the year-draining torture machine in The Princess Bride. This study measures a casual lifestyle correlation — people who watch a lot of TV, on average, die younger than those who do not.

This seems to make sense – it is not TV watching that is the real issue but rather sitting around a lot, which is related to TV watching. This was echoed in the HealthDay story the EW post refers to:

But other experts cautioned that the study did not show that TV watching caused people to die sooner, only that there was an association between watching lots of TV and a shorter lifespan.

But I wonder if this is more of a conceptual issue that an analysis issue on the part of the original researchers. While I can’t access the original article, here is part of the abstract that sheds light on the issue:

Methods The authors constructed a life table model that incorporates a previously reported mortality risk associated with TV time. Data were from the Australian Bureau of Statistics and the Australian Diabetes, Obesity and Lifestyle Study, a national population-based observational survey that started in 1999–2000. The authors modelled impacts of changes in population average TV viewing time on life expectancy at birth.

Results The amount of TV viewed in Australia in 2008 reduced life expectancy at birth by 1.8 years (95% uncertainty interval (UI): 8.4 days to 3.7 years) for men and 1.5 years (95% UI: 6.8 days to 3.1 years) for women. Compared with persons who watch no TV, those who spend a lifetime average of 6 h/day watching TV can expect to live 4.8 years (95% UI: 11 days to 10.4 years) less. On average, every single hour of TV viewed after the age of 25 reduces the viewer’s life expectancy by 21.8 (95% UI: 0.3–44.7) min. This study is limited by the low precision with which the relationship between TV viewing time and mortality is currently known.

Conclusions TV viewing time may be associated with a loss of life that is comparable to other major chronic disease risk factors such as physical inactivity and obesity.

Some key parts of this:

1. This was done using life table models, not correlations. Without seeing the full article, it is hard to know exactly what the researchers did. Did they simply calculate a life table (see an example in 7.2 here) or did they run a model that included other independent variables?

2. Their confidence intervals are really wide. For example, the amount of TV watched in 2008 could only shorten someone’s life by 8.7 days, hardly a substantively significant amount over the course of a lifetime. Watching 6 hours a day on average (compared to those who watch no TV), could live just 11 minute shorter lives.

3. The abstract suggests there is “low precision” because this link hasn’t been studied before. If this is true, then we need a lot more science on the topic and more data. This article, then, becomes an opening or early study on the topic and is not the “definitive” study.

4. The conclusion section says “may be associated with a loss of life that is comparable to other major chronic disease risk factors such as physical inactivity and obesity.” The key word here is “may.” This might simply be an academic qualification but it is an important distinction between saying “proved” (how the public might want to interpret it).

Here is my guess at what happened: media reports (or perhaps even a press release) about the study were a lot more strident about these results than the researchers themselves. In fact, here is a piece from the HealthDay piece that suggests this may be the case:

Researchers in Australia found that people who averaged six hours a day of TV lived, on average, nearly five years less than people who watched no TV.

The emphasis here is on the average, not necessarily the confidence interval. This would be like reporting poll results that say a candidate leads by 6 over an opponent but forgetting to mention that the margin of error (a confidence interval) is 5.9.

What the HealthDay report should include: comments from the researchers themselves explaining the work. Interestingly, the story quickly suggests that other researchers say there are other factors at work but we never hear from the original researchers outside of a few pieces lifted from the study. Without the proper context, a study can become a “shock headline” used by media sites to drive traffic.

I do have to ask: does Entertainment Weekly have a vested interest in debunking a study like this since they are in the business of reviewing television shows and channels?