Can religion not be fully studied with surveys or do we not use survey results well?

In a new book (which I have not read), sociologist Robert Wuthnow critiques the use of survey data to explain American religion:

Bad stats are easy targets, though. Setting these aside, it’s much more difficult to wage a sustained critique of polling. Enter Robert Wuthnow, a Princeton professor whose new book, Inventing American Religion, takes on the entire industry with the kind of telegraphed crankiness only academics can achieve. He argues that even gold-standard contemporary polling relies on flawed methodologies and biased questions. Polls about religion claim to show what Americans believe as a society, but actually, Wuthnow says, they say very little…

Even polling that wasn’t bought by evangelical Christians tended to focus on white, evangelical Protestants, Wuthnow writes. This trend continues today, especially in poll questions that treat the public practice of religion as separate from private belief. As the University of North Carolina professor Molly Worthen wrote in a 2012 column for The New York Times, “The very idea that it is possible to cordon off personal religious beliefs from a secular town square depends on Protestant assumptions about what counts as ‘religion,’ even if we now mask these sectarian foundations with labels like ‘Judeo-Christian.’”…

These standards are largely what Wuthnow’s book is concerned with: specifically, declining rates of responses to almost all polls; the short amount of time pollsters spend administering questionnaires; the racial and denominational biases embedded in the way most religion polls are framed; and the inundation of polls and polling information in public life. To him, there’s a lot more depth to be drawn from qualitative interviews than quantitative studies. “Talking to people at length in their own words, we learn that [religion] is quite personal and quite variable and rooted in the narratives of personal experience,” he said in an interview…

In interviews, people rarely frame their own religious experiences in terms of statistics and how they compare to trends around the country, Wuthnow said. They speak “more about the demarcations in their own personal biographies. It was something they were raised with, or something that affected who they married, or something that’s affecting how they’re raising their children.”

I suspect such critiques could be leveled at much of survey research: the questions can be simplistic, the askers of the questions can have a variety of motives and skills in developing useful survey questions, and the data gets bandied about in the media and public. Can surveys alone adequately address race, cultural values, politics views and behaviors, and more? That said, I’m sure there are specific issues with surveys regarding religion that should be addressed.

I wonder, though , if another important issue here is whether the public and the media know what to do with survey results. This book review suggests people take survey findings as gospel. They don’t know about the nuances of surveys or how to look at multiple survey questions or surveys that get at similar topics. Media reports on this data are often simplistic and lead with a “shocking” piece of information or some important trend (even if the data suggests continuity). While more social science projects on religion could benefit from mixed methods or by incorporating data from the other side (whether quantitative or qualitative), the public knows even less about these options or how to compare data. In other words, surveys always have issues but people are generally innumerate in knowing what to do with the findings.

Study tracking Baltimore kids with hundreds of interviews over 20 years shows rising out of poverty is hard

A recently published long-term study of Baltimore kids shows that escaping poverty is a difficult task:

First, its impressive length and scope; Alexander and his colleague, Doris Entwisle, devoted their careers to the project, conducting interviews of 790 children and their relatives over more than two decades. Alexander retires this summer as chair of the Hopkins sociology department; Entwisle died last year of cancer. (Linda Olson, a Hopkins instructor and researcher in the School of Education, is the third author of the report, published this month by the Russell Sage Foundation as a book titled “The Long Shadow: Family Background, Disadvantaged Urban Youth, and the Transition to Adulthood.”)…

Only 4 percent of the children from low-income families ended up with a college degree by the time they were 28. Kids from a middle-class or affluent background did 10 times better than that, with 45 percent getting a diploma.

Nearly half of the 1982 first-graders ended up at the same socioeconomic level as their parents.

By the time they were young adults, only 33 children had moved from low-income families to the high-income bracket. That doesn’t mean they didn’t want to, Alexander told me. It means they faced too many obstacles.

Stories of rising from humble origins may be popular but they are not the common pattern. Indeed, such rags-to-riches examples tend to be based on anecdotes while a project like this highlights large-scale interview data.

Google says their creative interview questions didn’t predict good workers…so why ask them?

Google announced yesterday that their creative and odd interview questions didn’t help them understand who was going to be a good worker. So, why did they ask them?

“We found that brainteasers are a complete waste of time,” Laszlo Bock, senior vice president of people operations at Google, told the New York Times. “They don’t predict anything. They serve primarily to make the interviewer feel smart.”

A list of Google questions compiled by Seattle job coach Lewis Lin, and then read by approximately everyone on the entire Internet in one form or another, included these humdingers:

  • How much should you charge to wash all the windows in Seattle?
  • Design an evacuation plan for San Francisco
  • How many times a day does a clock’s hands overlap?
  • A man pushed his car to a hotel and lost his fortune. What happened?
  • You are shrunk to the height of a nickel and your mass is proportionally reduced so as to maintain your original density. You are then thrown into an empty glass blender. The blades will start moving in 60 seconds. What do you do?

Bock says Google now relies on more quotidian means of interviewing prospective employees, such as standardizing interviews so that candidates can be assessed consistently, and “behavioral interviewing,” such as asking people to describe a time they solved a difficult problem. It’s also giving much less weight to college grade point averages and SAT scores.

The suggestion here is that these were more about the interviewer than the interviewee. Interesting. This is just speculation but here are other potential reasons for asking such questions.

1. They really thought these questions would be a good filter – but they learned better later. Was this initial idea based on research? Experience? Anecdotes? Or did this just sort of happen one time and it seemed to work so it continued? For a company that is all about data and algorithms, it would be interesting to know whether this interviewing practice was based on data.

2. Perhaps Google is trying to project a certain image to potential employees: “We are a place that values this kind of thinking.” The interview at Google isn’t just a typical interview; it is an experience.

3. They wanted to be to the wider public as a place that asked these kind of intimidating/interesting (depending on your point of view) questions. And this image is tied to social status: “Google does something in their interviews that others don’t! They must know something.” Were these questions all part of a larger branding strategy? It would be interesting to know how long they have thought the questions didn’t predict good workers. What does it say about the company now if they are moving on to other methods and more “quotidian”/pedestrian/boring interviewing approaches?

The Guardian starts a series about London riots written by sociologists who have talked with those involved

Now that some time has passed since the London riots (earlier posts here, here, and here), The Guardian is beginning a series consisting of sociologists talking about what they have learned about the riots from talking with those who were involved. The series starts here:

Many interviewees identified deprivation and inequality as root issues. Some spoke about the lack of work opportunities and access to education, as well as the EMA cuts. Some believed that getting an education was the key to the golden gate, but a year after graduation they were still struggling to find work. For others, also out of work, a university degree had never been on the cards.

But much of the anger was directed at the police. Young people spoke of incessant stop-and-search accompanied by rudeness, arrogance and racism. Some young people talked of Duggan’s death not as a unique injustice, but as yet another example of police murder. Young people spoke of the riots as a means of “sticking two fingers up” at the authorities, and for a couple of nights relishing having the upper-hand…

Finally, many young people talked about the riots as a consequence of the anger and frustration felt at not seeing a future. Unable to see education, jobs and pensions on their horizon, some explained how they sought pleasure in consumerism. But while those they looked up to accessed and displayed these objects freely, for young people they were often out of reach. As consumers first and foremost, the inability to shop made them feel unfulfilled and lacking in self respect. In some places the signs of these divides were part of the architecture around them – the upward mobility of the cityscapes of global capitalism looked increasingly remote.

Some young people hoped the government would hear the riots as a call to close the gap between the haves and the have-nots. But they didn’t think this likely.

I’ll be interested to read the subsequent pieces in this series. While the riots were often narrated by outsiders (particularly here in the United States),

interviews with those involved could reveal the internal motivations that led to actions that outsiders would see as out of line. Such broad and faraway views can be interpreted in different ways as Darnell Hunt illustrated how race influenced how people viewed news coverage of the Los Angeles riots of 1992. I hope we can soon to get to a point where we have some harder data, beyond even the vague statements from interviews offered in this opening piece. While the public may not want to hear the motivations of those who participated (particularly since it may be interpreted as justifying their actions), this is needed information in order to alter structural conditions and help prevent and/or predict future riots.

Indeed, a criminologist also thinks we need more “credible research” regarding the riots. (A side note: I use Newburn’s 2001 piece on evaluation research in my research methods class.)

High rate of arrests among NFL players?

Going into the Super Bowl, everyone knows about the legal issues Ben Roethlisberger has faced in recent years. But one sociologist has found that behavior that breaks the law is not unknown to NFL players. One website suggests that sociologist Eric Carter found “nearly 35% of all players in the league have been arrested.” Elsewhere, Carter goes into more detail about why so many NFL players are arrested at some point and how religion could help players deal with anomie:

Eric has conducted over 100 interviews with NFL players, some who have led happy and well-adjusted lives but also with many who have not.  We talk about the typical pressures that a professional player faces, coming into sudden fame and fortune.  Prof. Carter brings the research ideas of Emile Durkheim, particulary “social anomie,” to bear on what a number of these athletes face when moving into the professional ranks.  The sudden change in lifestyle combined with intense pressures to perform often leave many of them unhappy, confused and susceptible to all sorts of deviant behavior (some of which makes the news).  We talk then about the role of religion in helping players cope with these changes.  Our discussion looks at what factors might help players make adjustments to their new environments, including: a religious upbringing; the support networks they have access to at college; and religious role models in the locker room.

More details about Carter’s study of NFL players can be found here. Although this is a small sample of 104 players (there are at least 1440 players in the NFL each year – 45 players on 32 rosters), Carter found that 33 of the players had been arrested (31.7%). And Carter wrote a book, Boys Gone Wild, based on his study.

What is interesting about this is that the NFL seems to avoid scrutiny in the public eye about this. Whereas baseball stars are vilified for cheating, NFL players are regularly arrested (if this arrest rate holds true across the NFL – and even if it was really 10-20% lower, it still is a decent number) and the popularity of the NFL has continued to grow. Even with players like Roethlisberger or Rae Carruth or Michael Vick or Ray Lewis or Donte Stallworth or Marvin Harrison getting into trouble, this sort of news gets overwhelmed by the behemoth that is NFL entertainment.

In a more recent interview, Carter talks about how the NFL is able to keep this information out of the public eye:

“We see a lot of what goes on, because of the media,” Carter said. “But I was amazed at how much goes on that isn’t picked up — how powerful the NFL is in combating some of the potential bad media. I couldn’t believe how many guys contemplated suicide or attempted it, or were that unhappy with their lives that they engaged in these self-destructive behaviors.”

Carter found that 32 percent of the players he interviewed had been arrested after they entered the league — and others said they often evaded arrest by dispensing autographs to star-struck police officers — and nearly half described themselves as unhappy people.

“Fifty percent? That’s a big number,” he said, especially when you consider that these are young men who make on average more than $1 million a year to play football, and many of them much more than that.

“It just goes against our contemporary American conceptions of what happiness is. They have it all. They have the wealth, the fame, the power, the status — all of those things that many people equate with a happy life.”

Perhaps the NFL is able to bury these stories or minimize them. Or perhaps the American public doesn’t want to face this kind of information or thinks the athletes are compensated enough and can deal with the problems on their own.

Combining quantative and qualitative data collection on the Internet

I’ve quickly seen some recent mentions of a new project out of Princeton called All Our Ideas. Here is how the creators describe the project:

All Our Ideas is a research project to develop a new form of social data collection that combines the best features of quantitative and qualitative methods. Using the power of the web, we are creating a data collection tool that has the scale, speed, and quantification of a survey while still allowing for new information to “bubble up” from respondents as happens in interviews, participant observation, and focus groups.

Of course, one of the problems with surveys is that they force respondents to fit their responses to the questions that are asked. If you ask bad questions, you get bad results or if you don’t provide the options respondents want, you don’t really get the kind of data you want. Qualitative data, on the other hand, tends to be limited to a smaller sample because it takes more time to interview people or conduct focus groups.

I will be very curious to see what emerges out of this website.