Tag Archives: methods

Capturing life in a two minute window for social media – or for research

Posted on by

The increasingly popular social media platform BeReal gives users a two minute window in which to post each day:

Photo by Hannah Nelson on Pexels.com

Founded in January 2020, BeReal is advertised as “an authentic, spontaneous, and candid social network.” It’s an app that sends all users a notification at a seemingly random time in the day and gives them two minutes to post a photo from their front- and back-facing cameras, capturing the scene around them right at that moment. Users can always post late—though the app will then tell on them—but they can’t see what their friends have posted until they do.

BeReal co-founder Alexis Barreyat’s professed goal in creating the app was to foster “genuine” interactions online, the company said in marketing materials, “in response to a feeling that current social apps are doing everything else but connecting us with our friends and family.” But the real conceit here is that most of the time, you’re probably doing something incredibly mundane like studying or running errands, so the app deglamorizes our lives as seen on Instagram.

Without getting into whether it is possible long-term to have a social media platform that operates this way, I had another thought when reading this description: this sounds like a research protocol. Researchers are examining a particular topic, they ask participants to download an app, and at a random time each day the participant is asked some questions. Indeed, I have read about a research project that did something very similar. And it led to good data and published work.

In general, I am in favor of methods that help us better get at what people do as part of normal life or when they are alone. It is one thing to ask people to report on these times or to observe people doing these things. It is another to stop them briefly in the moment to report what they are doing and/or experiencing.

Researchers would need a lot of participants to collect meaningful data. Or, perhaps they would check in randomly multiple times a day. Imagine an research aggregator app that would allow people to respond quickly to multiple projects daily. However this ends up working, I suspect pushing the research closer to what people are doing in the moment could only help us get at what happens moment to moment in life.

“So what are the rules of ethnography, and who enforces them?”

Posted on by

A journalist looking into the Goffman affair discusses the ethics of ethnography:

To find out, I called several sociologists and anthropologists who had either done ethnographic research of their own or had thought about the methodology from an outside perspective. Ethnography, they explained, is a way of doing research on groups of people that typically involves an extended immersion in their world. If you’re an ethnographer, they said, standard operating procedure requires you to take whatever steps you need to in order to conceal the identities of everyone in your sample population. Unless you formally agree to fulfill this obligation, I was told, your research proposal will likely be blocked by the institutional review board at your university…

The frustration is not merely a matter of academics resenting oversight out of principle. Many researchers think the uncompromising demand for total privacy has a detrimental effect on the quality of scholarship that comes out of the social sciences—in part because anonymization makes it impossible to fact-check the work…

According to Goffman, her book is no less true than Leovy’s or LeBlanc’s. That’s because, as she sees it, what sociologists set out to capture in their research isn’t truths about specific individuals but general truths that tell us how the world works. In her view, On the Run is a true account because the general picture it paints of what it’s like to live in a poor, overpoliced community in America is accurate.

“Sociology is trying to document and make sense of the major changes afoot in society—that’s long been the goal,” Goffman told me. Her job, she said, as a sociologist who is interested in the conditions of life in poor black urban America, is to identify “things that recur”—to observe systemic realities that are replicated in similar neighborhoods all over the country. “If something only happens once, [sociologists are] less interested in it than if it repeats,” she wrote to me in an email. “Or we’re interested in that one time thing because of what it reveals about what usually happens.” This philosophy goes back to the so-called Chicago school of sociology, Goffman added, which represented an attempt by observers of human behavior to make their work into a science “by finding general patterns in social life, principles that hold across many cases or across time.”…

Goffman herself is the first to admit that she wasn’t treating her “study subjects” as a mere sample population—she was getting to know them as human beings and rendering the conditions of their lives from up close. Her book makes for great reading precisely because it is concerned with specifics—it is vivid, tense, and evocative. At times, it reads less like an academic study of an urban environment and more like a memoir, a personal account of six years living under extraordinary circumstances. Memoirists often take certain liberties in reconstructing their lives, relying on memory more than field notes and privileging compelling narrative over strict adherence to the facts. Indeed, in a memoir I’m publishing next month, there are several moments I chose to present out of order in order to achieve a less convoluted timeline, a fact I flag for the reader in a disclaimer at the front of the book.

Not surprisingly, there is disagreement within the discipline of sociology as well as across disciplines about how ethnography could and should work. It is a research method that requires so much time and personal effort that it can be easy to tie to a particular researcher and their laudable steps or mistakes. This might miss the forest for the trees; I’ve thought for a while that we need more discussion across ethnographies rather than seeing them as either the singular work on the subject. In other words, does Goffman’s data line up with what others have found in studying race, poor neighborhoods, and the criminal justice system? And if there are not comparisons to make with Goffman’s work, why aren’t more researchers wrestling with the same topic?

Additionally, this particular discussion highlights longstanding tensions in sociology: qualitative vs. quantitative data (with one often assumed to be more “fact”); “facts” versus “interpretation”; writing academic texts versus books for more general audiences; emphasizing individual stories (which often appeals to the public) versus the big picture; dealing with outside regulations such as IRBs that may or may not be accustomed to dealing with ethnographic methods in sociology; and how to best do research to help disadvantaged communities. Some might see these tensions as more evidence that sociology (and other social sciences) simply can’t tell us much of anything. I would suggest the opposite: the realities of the social world are so complex that these tensions are necessary in gathering and interpreting comprehensive data.

Science problem: study says there is not enough information in methods sections of science articles to replicate

Posted on by

A new study suggests the methods sections in science articles are incomplete, making it very difficult to replicate the studies:

Looking at 238 recently published papers, pulled from five fields of biomedicine, a team of scientists found that they could uniquely identify only 54 percent of the research materials, from lab mice to antibodies, used in the work. The rest disappeared into the terse fuzz and clipped descriptions of the methods section, the journal standard that ostensibly allows any scientist to reproduce a study.

“Our hope would be that 100 percent of materials would be identifiable,” said Nicole A. Vasilevsky, a project manager at Oregon Health & Science University, who led the investigation.

The group quantified a finding already well known to scientists: No one seems to know how to write a proper methods section, especially when different journals have such varied requirements. Those flaws, by extension, may make reproducing a study more difficult, a problem that has prompted, most recently, the journal Nature to impose more rigorous standards for reporting research.

“As researchers, we don’t entirely know what to put into our methods section,” said Shreejoy J. Tripathy, a doctoral student in neurobiology at Carnegie Mellon University, whose laboratory served as a case study for the research team. “You’re supposed to write down everything you need to do. But it’s not exactly clear what we need to write down.”

A new standard could be adopted across journals and subfields: enough information has to be given in the methods section for another scientist to replicate the study. Another advantage of this might be that it pushes authors to try to read their paper from the perspective of outsiders who are looking at the study for the first time.

I wonder how well sociology articles would fare in this analysis. Knowing everything needed for replication can get voluminous or technical, depending on the work that went into collecting the data and then getting it ready for analysis. There are a number of choices along the way that add up.

New study on American church attendance: a 10-18 percent gap between what people say versus what they actually do

Posted on by

The United States is consistently cited as a religious nation. The contrast is often drawn with a number of European nations where church attendance is usually said to be significantly lower than the American rate of about 40-45% of Americans attending on a regular basis. These figures have driven several generations of sociologists to debate the secularization thesis and why the American religious landscape is different.

But what if Americans overstate their church attendance on surveys and in reality, do attend church on a rate similar to European nations? A new study based on time diary data suggests this is the case:

While conventional survey data show high and stable American church attendance rates of about 35 to 45 percent, the time diary data over the past decade reveal attendance rates of just 24 to 25 percent — a figure in line with a number of European countries.

America maintains a gap of 10 to 18 percentage points between what people say they do on survey questions, and what time diary data says they actually do, Brenner reports. The gaps in Canada resemble those in America, and in both countries, gaps are both statistically and substantively significant…

“The consistency and magnitude of the American gap in light of the multiple sources of conventional survey data suggests a substantive difference between North America and Europe in overreporting.”

Given these findings, Brenner notes, any discussion of exceptional American religious practice should be cautious in using terms like outlier and in characterizing American self-reported attendance rates from conventional surveys as accurate reports of behavior. Rather, while still relatively high, American attendance looks more similar to a number of countries in Europe, after accounting for over-reporting.

A couple of thoughts about this:

1. This is another example where the research method used to collect data matters. Ask people about something on a survey and then compare that data to what people report in a time diary and it is not unusual to get differing responses. What exactly is going on here? Surveys ask people to consult their memory, a notoriously faulty source of information. Diaries have their own issues but supposedly are better at getting better information about daily or regular practices.

2. Even if church attendance data is skewed in the US, it doesn’t necessarily mean that America might still not be exceptional in terms of religion. Religiosity is made up of a number of factors including doctrinal beliefs, importance of religion in everyday life, membership in a religious congregation, the prevalence of other religious practices, and more. Church attendance is a common measure of religiosity but not the only one.

3. This is interesting data but it leads to another interesting question: why exactly would Americans overestimate their church attendance by this much? Since the time diary data from Europe showed a smaller gap, it suggests that Americans think they have something to gain by overestimating their church attendance. Perhaps Americans think they should say they attend church more – there is still social value and status attached to the idea that one attends church.