Tag Archives: social science

Don’t dismiss social science research just because of one fradulent scientist

Posted on by

Andrew Ferguson argued in early December that journalists fall too easily for bad academic research. However, he seems to base much of his argument on the actions of one fraudulent scientist:

Lots of cultural writing these days, in books and magazines and newspapers, relies on the so-called Chump Effect. The Effect is defined by its discoverer, me, as the eagerness of laymen and journalists to swallow whole the claims made by social scientists. Entire journalistic enterprises, whole books from cover to cover, would simply collapse into dust if even a smidgen of skepticism were summoned whenever we read that “scientists say” or “a new study finds” or “research shows” or “data suggest.” Most such claims of social science, we would soon find, fall into one of three categories: the trivial, the dubious, or the flatly untrue.

A rather extreme example of this third option emerged last month when an internationally renowned social psychologist, Diederik Stapel of Tilburg University in the Netherlands, was proved to be a fraud. No jokes, please: This social psychologist is a fraud in the literal, perhaps criminal, and not merely figurative, sense. An investigative committee concluded that Stapel had falsified data in at least “several dozen” of the nearly 150 papers he had published in his extremely prolific career…

But it hardly seems to matter, does it? The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior…

Who cares? The experiments are preposterous. You’d have to be a highly trained social psychologist, or a journalist, to think otherwise. Just for starters, the experiments can never be repeated or their results tested under controlled conditions. The influence of a hundred different variables is impossible to record. The first group of passengers may have little in common with the second group. The groups were too small to yield statistically significant results. The questionnaire is hopelessly imprecise, and so are the measures of racism and homophobia. The notions of “disorder” and “stereotype” are arbitrary—and so on and so on.

Yet the allure of “science” is too strong for our journalists to resist: all those numbers, those equations, those fancy names (say it twice: the Self-Activation Effect), all those experts with Ph.D.’s!

I was afraid that the actions of one scientist might taint the work of many others.

But there are a couple of issues here and several are worth pursuing:

1. The fact that Stapel committed fraud doesn’t mean that all scientists do bad work. Ferguson seems to want to blame other scientists for not knowing Stapel was committing fraud – how exactly would they have known?

2. Ferguson doesn’t seem to like social psychology. He does point to some valid methodological concerns: many studies involve small groups of undergraduates. Drawing large conclusions from these studies is difficult and indeed, perhaps dangerous. But this isn’t all social psychology is about.

2a. More generally, Ferguson could be writing about a lot of disciplines. Medical research tends to start with small groups and then decisions are made. Lots of research, particularly in the social sciences, could be invalidated if Ferguson was completely right. Ferguson really would suggest “Most such claims of social science…fall into one of three categories: the trivial, the dubious, or the flatly untrue.”?

3. I’ve said it before and I’ll say it again: journalists need more training in order to understand what scientific studies mean. Science doesn’t work in the way that journalists suggests where there is a steady stream of big findings. Rather, scientists find something and then others try to replicate the findings in different settings with different populations. Science is more like an accumulation of evidence than a lot of sudden lightning strikes of new facts. One small study of undergraduates may not tell us much but dozens of such experiments among different groups might.

4. I can’t help but wonder if there is a political slant to this: what if scientists were reporting positive things about conservative viewpoints? Ferguson complains that measuring things like racism and homophobia are difficult but this is the nature of studying humans and society. Ferguson just wants to say that it is all “arbitrary” – this is simply throwing up our hands and saying the world is too difficult to comprehend so we might as well quit. If there isn’t a political edge here, perhaps Ferguson is simply anti-science? What science does Ferguson suggest is credible and valid?

In the end, you can’t dismiss all of social psychology because of the actions of one scientist or because journalists are ill-prepared to report on scientific findings.

h/t Instapundit

Living Earth Simulator to model social world

Posted on by

Here is an interesting project, the Living Earth Simulator, that hopes to take a lot of data and come to conclusions about social life:

Described as a “knowledge collider,” and now with a pledge of one billion euros from the European Union, the Living Earth Simulator is a new big data and supercomputing project that will attempt to uncover the underlying sociological and psychological laws that underpin human civilization. In the same way that CERN’s Large Hadron Collider smashes together protons to see what happens, the Living Earth Simulator (LES) will gather knowledge from a Planetary Nervous System (PNS — yes, really) to try to predict societal fluctuations such as political unrest, economic bubbles, disease epidemics, and so on.

Orchestrated by FuturICT, which is basically a consortium of preeminent scientists, computer science centers around the world, and high-power computing (HPC) installations, the Living Earth Simulator hopes to correlate huge amounts of data — including real-time sources such as Twitter and web news — and extant, but separate approaches currently being used by other institutions, into a big melting pot of information. To put it into scientific terms, the LES will analyze techno-socio-economic-environmental (!) systems. From this, FuturICT hopes to reveal the tacit agreements and hidden laws that actually govern society, rather than the explicit, far-removed-from-reality bills and acts that lawmakers inexorably enact…

The timing of EU’s billion-euro grant is telling, too. As you probably know, the European Union is struggling to keep the plates spinning, and the LES, rather handily, will probably be the most accurate predictor of economic stability in the world. Beyond money, though, it is hoped that the LES and PNS can finally tell us why humans do things, like watch a specific TV show, buy a useless gadget, or start a revolution.

Looking at the larger picture, the Living Earth Simulator is really an admission that we know more about the physical universe than the social. We can predict with startling accuracy whether an asteroid will hit Earth, but we know scant little about how society might actually react to an extinction-level event. We plough billions of dollars into studying the effects and extent of climate change, but what if understood enough of the psychology and sociology behind human nature to actually change our behavior?

I don’t know about the prospects of such a project but if the BBC is reporting on it, perhaps it has a future.

A couple of statements in the description above intrigue me:

1. The simulator will help uncover “the tacit agreements and hidden laws that actually govern society.” Do most social scientists think this is possible if we only had enough data and the right simulator?

2. The comparison between the natural and social sciences is telling. The portrayal here is that the natural sciences have come a lot further in studying nature than social scientists in studying human behavior. Is this true? Is this a fair comparison – natural systems vs. social systems? How much “unknown knowledge” is really in each realm?

3. The coding for this project must be immense.

4. The article makes no mention of utilizing social scientists to help develop this project and analyze the data though the group behind it does have some social scientists on board.

David Brooks: keep government funding for social science research

Posted on by

Last Thursday, David Brooks made a case for retaining government money for social science research:

Fortunately, today we are in the middle of a golden age of behavioral research. Thousands of researchers are studying the way actual behavior differs from the way we assume people behave. They are coming up with more accurate theories of who we are, and scores of real-world applications. Here’s one simple example:

When you renew your driver’s license, you have a chance to enroll in an organ donation program. In countries like Germany and the U.S., you have to check a box if you want to opt in. Roughly 14 percent of people do. But behavioral scientists have discovered that how you set the defaults is really important. So in other countries, like Poland or France, you have to check a box if you want to opt out. In these countries, more than 90 percent of people participate.

This is a gigantic behavior difference cued by one tiny and costless change in procedure.

Yet in the middle of this golden age of behavioral research, there is a bill working through Congress that would eliminate the National Science Foundation’s Directorate for Social, Behavioral and Economic Sciences. This is exactly how budgets should not be balanced — by cutting cheap things that produce enormous future benefits.

Here is what I think works in this column:

1. The examples are interesting and address important issues. I wish there were more people highlighting interesting research in such large venues.

2. The idea that a small research investment can have large results.

3. The reminder in the last paragraph: “People are complicated.”

Here is where I think this column could use some more work: why exactly should the government, as opposed to other organizations or sources, provide this money? (See a counterargument here.) Brooks could have made this case more clearly: there are a lot of social problems that affect our country and the government has the resources and clout to promote research. In certain areas, like poverty or public health, the government has a compelling interest in tackling these concerns as there are few other bodies that could handle the scope of these issues. Of course, many of these issues are politicized but that doesn’t necessarily mean that the government shouldn’t address these issues at all.

Applying evolutionary biology to the city

Posted on by

Biologist David Sloan Wilson has taken an interest in better understanding Binghamtom, New York. His lens: evolutionary biology.

Differences in prosociality, Wilson thought, should produce measurable outcomes — if not in reproductive success, perhaps in happiness, crime rates, neighbourhood tidiness or even the degree of community feeling expressed in the density of holiday decorations. “I really wanted to see a map of altruism,” he says. “I saw it in my mind.” And with a frisson of excitement, he realized that his models and experiments offered clues about how to intervene, how to structure real-world groups to favour prosociality. “Now is the implementation phase.” Evolutionary theory, Wilson decided, will improve life in Binghamton…

Binghamton is hard to love. Established in the early nineteenth century, it has long relied on big industry for its identity and prosperity — early on through the Endicott-Johnson Shoe Company and then through IBM, which was founded in the area. But the manufacturers mostly decamped in the 1990s, and since then the city has taken on an aimless, shabby air. Dollar stores and coin-operated laundries fill the gaps between dilapidated Victorian houses and massive brick-and-stone churches. A Gallup poll in March 2011 listed Binghamton as one of the five US cities least liked by its residents. “It is a town that knows it is badly in need of a revival,” says Wilson. Even its motto, ‘Restoring the pride’, speaks of a city clinging to its past and ashamed of its present…

So Wilson decided to see whether he could raise up the prosocial valleys by creating conditions in which cooperation becomes a winning strategy — in effect, hacking the process of cultural evolution. He set about this largely by instituting friendly competitions between groups. His first idea was a park-design project, in which neighbourhoods were invited to compete for park-improvement funds by creating the best plan.

But Wilson soon found out that field experiments in real cities can take on lives of their own: different neighbourhoods couldn’t get their acts together on the same schedule, so the competition aspect largely disappeared. Instead, he is now working on turning multiple park ideas into reality. The dog park is one. Another is Sunflower Park, the most advanced project to date, but still a sad, mainly empty lot surrounded by a chain-link fence. Children don’t spend a lot of time playing here. Undaunted, Wilson is raising funds and laying plans for a relaxing community space flush with trees and amenities. “In a year,” he says, “we will serve you a hot dog from the pavilion.”

The rest of the article describes how Wilson acts more like a social scientist, taking surveys, making observations, interacting with residents, trying to understand local religious congregations. Some of this discussion is amusing as it rehashes debates about how close researchers should get to research subjects – social scientists would describe it as participant observation.

This reminds me of some of the work of early sociologists such as Herbert Spencer and the Chicago School who based at least some of their ideas on biological principles. Spencer viewed society as being like an organism and the Chicago School viewed competition for space as a primary driver of urban development and action. But evolutionary thinking has generally faded away in sociology (outside of sociobiology). Could sociologists, and urban sociologists, again view evolutionary principles as a boon for the field or simply a distraction from the better work that is going on in the field? Wilson is also interested in the topics of altruism and prosociality, topics that have attracted the attention of more sociologists in recent years. It would be interesting to hear what happens when Wilson comes to some conclusions about social and city life and then presents them to social scientists.

The difference between a sociologist and a geologist, the “soft” and “hard” sciences

Posted on by

Comments about sociology can come from anywhere. See this example from a House member discussing FDA guidelines:

The most intense reaction was generated by a provision offered by Rep. Denny Rehberg (R-Mont.) that would block the FDA from issuing rules or guidance unless its decisions are based on “hard science” rather than “cost and consumer behavior.” The amendment would prevent the FDA from restricting a substance unless it caused greater harm to health than a product not containing the substance.

“The FDA is starting to use soft sciences in some considerations in the promulgation of its rules,” said Rehberg, who defined “hard science”, as “perceived as being more scientific, rigorous and accurate” than behavioral and social sciences.

“I hate to try and define the difference between a psychiatrist and a psychologist, between a sociologist and a geologist, but there is clearly a difference,” he told the committee.

Three sets of comparisons are made here: between psychology and psychiatry, sociology and geology, and “hard” and “soft” science. I think it is pretty easy to make the first two distinctions, particularly between geology and sociology. But the third comparison seems a little strange: does Rehberg want to suggest that soft sciences are less true or that they matter less/are less valid for FDA decision making?

Overall, it sounds like Rehberg is suggesting that the “soft” sciences (psychology and sociology) are not as important in crafting FDA policies as the actual science that says whether certain products are good or bad for humans. But it seems somewhat silly to suggest that perceptions and behaviors shouldn’t influence policy decisions. A lot of legislation is driven by perceptions and values in addition to the actual influences in the physical world. Think about some of the major issues being discussed today such as the deficit or taxes: less of the conversation is about the actual impact on the country and more involves ideologies about who should be responsible for funding the government and what is the proper role and/or size of the government. One of the problems presented in this article is instructive: cigarettes are not illegal and yet government bodies are interested in limiting the consumption of them. Therefore, while menthol cigarettes may not be that much more harmful, if it is attractive to younger kids who then take smoking, why not regulate this? Of course, the smoking example is a loaded one and it would be hard to find someone who would suggest more smoking among teenagers is a good thing.

Based on this discussion, would either political party be willing to create legislation only based on “hard science” or is this only a suggestion when the “hard science” supports one’s existing viewpoint? Additionally, are there politicians out there who have publicly supported sociology rather than suggested it is a “soft” science?

James Q. Wilson on the difficulties of studying culture

Posted on by

In a long opinion piece looking at possible explanations for the reduction in crime in America, James Q. Wilson concludes by suggesting that cultural explanations are difficult to test and develop:

At the deepest level, many of these shifts, taken together, suggest that crime in the United States is falling—even through the greatest economic downturn since the Great Depression—because of a big improvement in the culture. The cultural argument may strike some as vague, but writers have relied on it in the past to explain both the Great Depression’s fall in crime and the explosion of crime during the sixties. In the first period, on this view, people took self-control seriously; in the second, self-expression—at society’s cost—became more prevalent. It is a plausible case.

Culture creates a problem for social scientists like me, however. We do not know how to study it in a way that produces hard numbers and testable theories. Culture is the realm of novelists and biographers, not of data-driven social scientists. But we can take some comfort, perhaps, in reflecting that identifying the likely causes of the crime decline is even more important than precisely measuring it.

I find it a little strange that a social scientist wants to leave culture to the humanities (“novelists and biographers”). This sounds like a traditional social science perspective: culture is a slippery concept that is difficult to quantify and make generalizations about. I can imagine this viewpoint from quantitatively minded social scientists who would ask, “where it the data?”

But there is a lot of good research regarding culture that utilizes data. Some of this data is fuzzier qualitative data that involves ethnographies and long interviews and observations. But other data regarding culture comes from more traditional data sources such as large surveys. And if you put together a lot of these data-driven studies, qualitative and quantitative, I think you could put together some hypotheses and ideas regarding American culture and crime. Perhaps all of this data can’t fit into a regression or this isn’t the way that crime is traditionally studied but that doesn’t mean we have to simply abandon cultural explanations and studies.

The predictive power of sociology and learning from the past

Posted on by

In recent  years, the predictive element of social science has been discussed by a few people: how much can we use data from the past to predict the future? In an interview with Scientific American, a mathematical sociologist who works at Yahoo! Labs talks about our predictive abilities:

A big part of your book deals with the problem of ignoring failures—a selective reading of the past to draw erroneous conclusions, which reminds me of the old story about the skeptic who hears about sailors who survived a shipwreck supposedly because they’d prayed to the gods. The skeptic asked, “What about the people who prayed and perished?”
Right—if you look at successful companies or shipwrecked people, you don’t see the ones who didn’t make it. It’s what sociologists call “selection on the dependent variable,” or what in finance is called survivorship bias. If we collected all the data instead of just some of it, we could learn more from the past than we do. It’s also like Isaiah Berlin’s distinction between hedgehogs and foxes. The famous people in history were hedgehogs, because when those people win they win big, but there are lots of failed hedgehogs out there.

Other scholars have pointed out that ignoring this hidden history of failures can lead us to take bigger risks than we might had we seen the full distribution of past outcomes. What other problems do you see with our excessive focus on the successful end of the distribution?
It causes us to misattribute the causes of success and failure: by ignoring all the nonevents and focusing only on the things that succeed, we don’t just convince ourselves that things are more predictable than they are; we also conclude that these people deserved to succeed—they had to do something right, otherwise why were they successful? The answer is random chance, but that would cause us to look at them in a different light, and changes the nature of reward and punishment.

Interesting material and Watts’ just published book (Everything Is Obvious: *Once You Know the Answer)  sounds worthwhile. There are also some interesting thoughts later in the interview about how information in digital social networks doesn’t really get passed along through influential people.

I haven’t seen too much discussion within sociology about predictive abilities: how much do we suffer from these blind spots that Watts and others point out?

(As a reminder, Nassim Taleb, in his book Black Swan, has also written well on this subject.)

A stronger market for sociological themes, due in part to the rise of the “Gladwellian” genre

Posted on by

While considering David Brook’s new book The Social Animal (some earlier thoughts here), a reviewer provides some context by explaining why books invoking social science, including sociology, have been more popular recently:

The public appetite for books on social science was weak for decades. In the 1980s, particle physicists and cosmologists like Stephen Hawking learned to cut out the equations and reached a big audience. But in the 1990s, interest shifted to sociology and psychology. Steven Pinker wrote about the evolution of the mind, and Malcolm Gladwell’s “The Tipping Point” signaled a tipping point itself by scaling the best-seller lists and staying there for 10 years (and counting).

Mr. Gladwell’s ability to create page-turners out of material from the Journal of Personality and Social Psychology cast a long shadow over the genre. “Gladwellian” would not have been listed on the Word Exchange in 1986, but if you had invested at its IPO in the early 2000s you would have earned a tidy sum by now. Authors in this genre now labor to find Gladwellian stories and characters to vivify the theories and studies that support their counterintuitive insights. Sometimes they focus on the stories to the exclusion of the studies, a practice that makes it easy to reach pleasing but unsound conclusions.

Gladwell’s use of sociological ideas and themes earned him the first award for “Excellence in the Reporting of Social Issues” in 2007 from the American Sociological Association.

While Gladwell certainly wasn’t the first to write such books, he certainly seems to have spawned a number of (often inferior) imitators. The “Gladwellian” genre goes something like this: take a common facet of life, explain why it matters, and then winsomely weave together both established bodies of scientific research with compelling stories. Since David Brooks seems to be taking a slightly different approach by developing two characters to carry his narrative, does this mean Brooks is trying to deliberately break out of this genre by returning to “the 18th-century didactic narrative”? Just how many “Gladwellian” books or articles can the market bear?

But someone could do a much deeper analysis of this. While Gladwell may have a good presentation, what changed in American society such that his books would find a receptive audience? If physics ruled the day in the 1980s, why the shift to the social sciences in the 1990s? Was this some response to a booming economy or the end of the Cold War or a number of new discoveries and exciting theories in the social sciences around this time?

Where are the social scientists to explain the global warming debate?

Posted on by

Amidst all of the political discussions regarding climate change and global warming, one social scientist suggests a sociological analysis of this public issue has been lacking:

But something is missing: academic explanations of why people flout reams of scientific conclusions, bristle at the notion of cutting carbon and regard climate change as a sneaky liberal plot.

“The social sciences are glaringly missing,” says Andrew Hoffman, an expert on the sociological aspects of environmental policies at the University of Michigan, for which he’s researching climate denial. “That leaves out critical questions about the cultural dimensions of both defining the problem and finding solutions.”

He provides unvarnished reasons for that. One concerns his colleagues’ dismissal of the conservative movement. They deny the deniers, he seems to say, by tending to “ignore the far right.” More broadly, social scientists — like sociologists, psychologists and communication researchers — are generally disengaged from public policy debates.

The story goes on to suggest that some research suggests that this debate may be similar to the debate over abortion: both sides attempt to frame the issue and then influence enough lawmakers to make their side heard. This seems like easy pickings for sociologists interested in social problems. Notwithstanding the science, how have both the supporters and skeptics’ movement been formed, framed, and publicized?

If this social scientist is correct, this means there are some real opportunities for sociologists to provide some overarching analysis of this important public debate.

Looking at the educationally rich Washington D.C. metro area

Posted on by

Based on recently released figures from the 2009 American Community Survey, the Washington Post describes the Washington D.C. metropolitan area, the most educated region in the country:

The data, drawn from questionnaires, underscore the academic primacy of the Washington area: 47 percent of adults – or nearly 2 million – hold bachelor’s degrees, the highest rate among the nation’s large urban areas. Six of the 10 best-educated U.S. counties are within commuting distance of the District.

Half of all degrees in the region are in natural and social sciences, well above the national average, under a broad Census Bureau definition of “science” that includes everything from nuclear physics to sociology. That’s a starting point for looking at variations within the region: So-called hard sciences reign beyond the Beltway, soft ones within.

There are some other interesting facts in here as well, such as the finding that the proportion of people of business degrees in the region is lower than the national average.

What I would be interested to know is how these statistics change the day-to-day lived experience of an average resident of the region. Are there more civic events with more involvement from nearby residents? Is there a large perceived gap between the lives of those who are well educated and those with less education? Do people strike up deeper conversations at any gathering? Does this concentration of people lead to more interdisciplinary work and research?