Good data is foundational to doing good sociological work

I’ve had conversations in recent months with a few colleagues outside the discipline about debates within sociology over the work of ethnographers like Alice Goffman, Matt Desmond, and Sudhir Venkatesh. It is enlightening to hear how outsiders see the disagreements and this has pushed me to consider more fully how I would explain the issues at hand. What follows is my one paragraph response to what is at stake:

In the end, what separates the work of sociologists from perceptive non-academics or journalists? (An aside: many of my favorite journalists often operate like pop sociologists as they try to explain and not just describe social phenomena.) To me, it comes down to data and methods. This is why I enjoy teaching both our Statistics course and our Social Research course: undergraduates rarely come into them excited but they are foundational to who sociologists are. What we want to do is have data that is (1) scientific – reliable and valid – and (2) generalizable – allowing us to see patterns across individuals and cases or settings. I don’t think it is a surprise that the three sociologists under fire above wrote ethnographies where it is perhaps more difficult to fit the method under a scientific rubric. (I do think it can be done but it doesn’t always appear that way to outsiders or even some sociologists.) Sociology is unique in both its methodological pluralism – we do everything from ethnography to historical analysis to statistical models to lab or natural experiments to mass surveys – and we aim to find causal explanations for phenomena rather than just describe what is happening. Ultimately, if you can’t trust a sociologist’s data, why bother considering their conclusions or why would you prioritize their explanations over that of an astute person on the street?

Caveats: I know no data is perfect and sociologists are not in the business of “proving” things but rather we look for patterns. There is also plenty of disagreement within sociology about these issues. In a perfect world, we would have researchers using different methods to examine the same phenomena and develop a more holistic approach. I also don’t mean to exclude the role of theory in my description above; data has to be interpreted. But, if you don’t have good data to start with, the theories are abstractions.

Quick Review: Evicted

I recently read Matthew Desmond’s much discussed work Evicted: Poverty and Profit in the American City. Here are my thoughts on the ethnographic work.

  1. The book is certainly readable as he tells the stories of a number of tenants and landlords in the Milwaukee area. The plight of the tenants is striking and the landlords are also an interesting group (particularly Sherrena who wanted to tell her story). Of course, such readability may not impress some sociologists who prefer more scientific prose (and who complain about the work of Venkatesh or Goffman) but this should reach a broader public. The narratives have some summary data and causal explanations sprinkled in but the emphasis is on the stories.
  2. One of the more impressive features of this work is the quantitative data that it also draws on. This information is buried in the footnotes but Desmond also developed several quantitative datasets that helped (1) suggest his stories are not unusual and (2) provide the broader patterns for an issue that is not studied much in sociology.
  3. The biggest takeaway for me: the number of evictions that take place on a regular basis.
  4. The subject area – evictions – certainly needs more attention. I’ve read my share of work on affordable housing in the last decade but rarely did I see this issue mentioned. As Desmond notes, big cities have a sizable population of people who consistently have to move around due to evictions. Even if there were more housing units – and big cities are often tens of thousands of units short of affordable units – evictions make it difficult to establish roots and settle kids into schools. The final chapter – where Desmond discusses the broader issue and possible solutions – leads off nicely with this idea of a good physical home as the centerpiece of a thriving society.
  5. That said, how common is this issue in suburban areas? As poverty moves to the suburbs as do increasing numbers of minorities, I would expect that evictions are not limited just to larger cities.
  6. One area that gets less attention in this ethnography that may also prove worthwhile to explore further is the legal apparatus. Desmond follows one of the eviction squads and provides some insights into the court process but it would be interesting to hear more from judges (who from the book seem to work against the tenants – though they may just be following the law) as well as local officials (how do public officials respond to these situations).
  7. A second area is thinking about the intersections of race and class. Desmond hints at the influence of race: comparing the experiences of blacks on the North Side of Milwaukee versus whites on the South Side, comments from black and white tenants about the possibilities for living in the other’s neighborhoods, briefly discussing the race of landlords. However, there is a lot more here to unpack, especially given Desmond’s other work on race. Take the two main landlords in the book: one is white, the other black. The first has a more stand-offish approach (working through intermediaries) while the second is more directly involved with tenants. Both are in it for the money and seem to be doing well. How much does their race matter?

An enjoyable read and a work I could imagine using with undergraduates who often have little to no experience with housing issues. I look forward to looking at Desmond’s journal articles that also build on this ethnographic and quantitative data.

Researchers fact-checking their own ethnographic data

Toward the end of a long profile of sociologist Matthew Desmond is an interesting section regarding ethnographic methods:

Desmond has done an especially good job spelling out precisely how he went about his research and verified his findings, says Klinenberg. At the start of Evicted, an author’s note states that most of the events in the book took place between May 2008 and December 2009. Except where it says otherwise in the notes, Desmond writes, all events that happened between those dates were observed firsthand. Every quotation was “captured by a digital recorder or copied from official documents,” he adds. He also hired a fact-checker who corroborated the book by combing public records, conducting some 30 interviews, and asking him to produce field notes that verified a randomly selected 10 percent of its pages.

Desmond has been equally fastidious about taking himself out of the text. Unlike many ethnographic studies, including Goffman’s, his avoids the first person. He wants readers to react directly to the people in Evicted. “Ethnography often provokes very strong feelings,” he says. “So I wanted the book to do that. But not about me.”

Ethnographers should be more skeptical about their data, Desmond believes. In his fieldwork, for example, he saw women getting evicted at higher rates than men. But when he crunched the data, analyzing hundreds of thousands of court records, it turned out that was only the case in predominantly black and Latino neighborhoods. Women in white neighborhoods were not evicted at higher rates than men. The field had told him a half-truth.

Still, beyond acknowledging that the reception of Goffman’s book shaped his fact-checking, he will say nothing about the controversy. Even an old journalism trick — letting a silence linger, in the hope that an interviewee will fill it — fails to wring a quote from him. “This is such a good technique,” he says after a few seconds, “where you just kind of let the person talk.” Then he sips his Diet Coke, waiting for the next question.

This gets at some basic questions about what ethnography is. Should it be participant observation with a reflexive and involved researcher? Letting the research subjects speak for themselves with minimal interpretation? Should it involve fact-checking and verifying data? Each of these could have their merit and sociologists pursue different approaches. Contrasting the last two, for example, how people describe their own circumstances and understanding could be very important even if what is reported is not necessarily true. On the other hand, more and more ethnographies involve reflexive commentary from the researcher on how their presence and personal characteristics influenced the data collection and inteprretation.

It sounds to me like Desmond is doing some mixed methods work: starting with ethnographic data that he directly observes but then using secondary analysis (in the example above, using official records) to better understand both the micro level that he observed as well as the broader patterns. This means more work for each study but also more comprehensive data.