Where does the data on the number of Americans traveling for Thanksgiving come from?

It was widely reported this year that nearly 49 million Americans would be traveling for Thanksgiving this year. This data comes from AAA and here is the methodology according to their press release from November 15:

AAA’s projections are based on economic forecasting and research by IHS Markit. The London-based business information provider teamed with AAA in 2009 to jointly analyze travel trends during major holidays. AAA has been reporting on holiday travel trends for more than two decades. The complete AAA/IHS 2016 Thanksgiving holiday travel forecast can be found here.

When numbers like this are used in public and reported on by the media, I would guess many Americans expect these figures to be based on surveys. So what is this projection based on? Surveys (probably via phone calls)? Historical models (based on factors like gas prices and broader economic indicators)? Certain retail and tourism figures like hotel and airfare bookings?

What makes this more complicated is that AAA is an organization that could benefit from increased travel, particularly driving. And as they note on the press release, their organization can provide benefits to travelers:

AAA will rescue thousands of motorists this Thanksgiving AAA expects to rescue more than 370,000 motorists this Thanksgiving, with the primary reasons being dead batteries, flat tires and lockouts. AAA recommends that motorists check the condition of their battery and tires and pack emergency kits in their vehicles before heading out on a holiday getaway. Drivers should have their vehicles inspected by a trusted repair shop, such as one of the nearly 7,000 AAA Approved Auto Repair facilities across North America. Members can download the AAA Mobile app, visit AAA.com or call 1-800-AAA-HELP to request roadside assistance.

This does not necessarily mean that the data is inaccurate. At the same time, it would help to make the methodology of their projections available.

Another thought: are Americans helped or hindered by these broad projections of holiday travel? If you are traveling, does news like this change your plans (i.e., leave earlier)? If AAA projects more drivers, do traffic delays increase (such as on the 405 in Los Angeles)? If the BBC links the incidents, perhaps people take these figures seriously…

Middle-class incomes have biggest year to year rise – with a catch

New data suggests middle-class incomes rose in 2015:

The incomes of typical Americans rose in 2015 by 5.2 percent, the first significant boost to middle-class pay since the end of the Great Recession and the fastest increase ever recorded by the federal government, the Census Bureau reported Tuesday.

In addition, the poverty rate fell by 1.2 percentage points, the steepest decline since 1968. There were 43.1 million Americans in poverty on the year, 3.5 million fewer than in 2014…

The 5.2 percent increase was the largest, in percentage terms, ever recorded by the bureau since it began tracking median income statistics in the 1960s. Bureau officials said it was not statistically distinguishable from five other previous increases in the data, most recently the 3.7 percent jump from 1997 to 1998.

Rising incomes are generally good. But, note the catch in the third paragraph cited above: officials cannot say that the 5.2% increase is definitively higher than several previous increases. Why not? The 5.2% figure is based on a sample that has a margin of error of at least 1.5% either way. The data comes from these Census instruments:

The Current Population Survey Annual Social and Economic Supplement was conducted nationwide and collected information about income and health insurance coverage during the 2015 calendar year. The Current Population Survey, sponsored jointly by the U.S. Census Bureau and U.S. Bureau of Labor Statistics, is conducted every month and is the primary source of labor force statistics for the U.S. population; it is used to calculate the monthly unemployment rate estimates. Supplements are added in most months; the Annual Social and Economic Supplement questionnaire is designed to give annual, national estimates of income, poverty and health insurance numbers and rates.

According to the report (page 6), the margin of error for the percent change in income from 2014 to 2015 is 1.6%. Incomes may have risen even more than 5.2%! Or, they may have risen at lower rates. See the methodological document regarding the survey instruments here.

The Census has in recent years moved to more frequent reports on key demographic measures. This produces data more frequently. One of the trade-offs, however, is that these estimates are not as accurate as the dicennial census which requires a lot more resources to conduct and is more thorough.

A final note: it is good that the margin of error is hinted at in the article on rising middle-class incomes. On the other hand, it is mentioned in paragraph 12 and the headline clearly suggests that this was a record year. Statistically speaking, this may or may not be the case.

“Sociology is alien to literature”

One reviewer of a new book suggests the retelling of personal experiences cannot be equated with sociology:

Ben Simon writes in the introduction, “I am not sure that my immigration experience is representative of the immigrants from Morocco.” But elsewhere he also writes: “To date, no attempt has been made to decipher the sociology of the Moroccan immigration. This book is a modest step in that direction.”

I object to Ben Simon’s sociological aspirations in this book. In his work as a journalist, he aimed his efforts in this direction, always doing so in an interesting and profound manner. But that is not the story here, because this is a different sort of literary undertaking. Someone who seeks to tell about himself has to first employ tools of emotion, sharing experiences and memories, allowing the reader to learn the process involved in consciousness-in-the-making: a private and personal consciousness, not a “sociology,” not the diagnosis of a society, not a creation of a portrait of something – but rather literature.

By its nature, an autobiography is first and foremost a literary text. And it is enough to think of Sartre’s “The Words” to understand this. Sociology, by virtue of the alienation that underlies its definition, in its critical sense of observation from the outside – is alien to literature. Being Moroccan is, in any event, much more complex, and so too are its immigrant experiences. It is enough for me to think about my “Moroccan” family, about its consciousness, about how it coped, about its relationship to religion and its immigration experiences.

Ben Simon sets out on a journey that traces the impressive path he has forged, the consolidation of his own perspective on reality, his emotions. But in “The Moroccans,” he feels a need to package this in “sociology.” Clearly there is a context, a “period,” a reflection of reality, but it is marginal; it is not the main thing.

Without reading the book, it is hard to know exactly what is going on here. It sounds like the author wants to extrapolate a bit from his own experiences to those of all Moroccan immigrants and the reviewer suggests he can’t speak for such a large group. This kerfuffle may also be about style; autobiographies and sociological works are often written differently with the emphasis of the first more on experiences and emotions and the second on larger generalizations, data, and theory.

This does hint at a larger issue in sociology and related disciplines where some research methods – particularly ethnography – allow for the mixing of researcher experience while still attempting to remain objective and connect the research to bigger issues in the field. This line can be quite blurry; see earlier issues raised about the work of Venkatesh or Goffman. Yet, it is an issue that is not going to go away as (1) insider information continues to be valuable and (2) some look to connect with different (i.e., non-academic) audiences with more literary styles.

Doing social science research in Madagascar

One researcher discusses undertaking research in Madagascar:

My colleagues and I, from the UK, the US and South Africa, feel frustrated. It is December 2014 and we have gathered at a jungle lodge in the highlands of Madagascar with 25 academics and postgraduate students from Antananarivo’s departments of sociology and communication to hash out the methodology for a large-scale research study. However, our research partners’ greater apparent interest in discussing theoretical issues is slowing us down. It is also tough for the interpreters, grappling with three-way simultaneous translation from Malagasy to English, French to English and English to French. The day reaches a low point when I hear through my headphones: “The real problem is situated somewhere between the problematic and the problematisation.”

We feel like prisoners in a jungle of theory. However, over the next few months, I come to realise that the lecture on Weber – and other diversions into Marxist, literary or linguistic theory – are not mere academic posturing. They are – to use development jargon – capacity-building. Unicef has asked our team to build the capacity of Antananarivo staff and students to conduct social research. We know how to design a quantitative and qualitative study, do the data analysis and write the report. But we know little about Madagascar: its culture and turbulent history, or how our Malagasy colleagues regard research. Their priority for the seminar is not to draft survey questionnaires but to build an equal, trusting research partnership…

According to the research design, a quantitative study (two questionnaires, with about 1,500 respondents for each) is to be conducted first, to highlight issues to be explored in the subsequent qualitative research. Unfortunately, the eastern floods and southern drought put the project several months behind schedule, and the Antananarivo qualitative research teams go into the field at about the same time as the quantitative research is being conducted, working in different communities. They emerge with hundreds of hours of focus group and interview transcripts and field notes, and it is a formidable task to merge them with the quantitative data.

Ultimately, common sense and pragmatism prevail. We use geographic and economic criteria to classify communities into four types: interior, sub-coastal, coastal and urban. Some interior communities are two days by zebu cart from the main dirt road; including them would lengthen the research and strain the budget. We reduce the long list of variables to be analysed. Our Antananarivo colleagues have a therapeutic 15-minute debate over whether coding – or, indeed, any attempt to organise human experience – is a colonial imposition. And then everyone goes back to work.

Doing quality research in first-world countries is difficult enough and yet working through the obstacles to doing good research in the developing world could lead to many positive consequences. It would be nice to see a follow-up article that shows what came of all these efforts.

The methodology of quantifying the cost of sprawl

A new analysis says sprawl costs over $107 billion each year – and here is how they arrived at that figure:

To get to those rather staggering numbers, Hertz developed a unique methodology: He took the average commute length, in miles, for America’s 50 largest metros (as determined by the Brookings Institution), and looked at how much shorter those commutes would be if each metro were more compact. He did this by setting different commute benchmarks for clusters of comparably populated metros: six miles for areas with populations of 2.5 million or below, and 7.5 miles for those with more than 2.5 million people. These benchmarks were just below the commute length of the metro with the shortest average commute length in each category, but still 0.5 miles within the real average of the overall category.

He multiplied the difference between the benchmark and each metro’s average commute length by an estimated cost-per-mile for a mid-sized sedan, then doubled that number to represent a daily roundtrip “sprawl tax” per worker, and then multiplied that by the number of workers within a metro region to get the area’s daily “sprawl tax.” After multiplying that by the annual number of workdays, and adding up each metro, he had a rough estimate of how much sprawl costs American commuters every year.

Then Hertz calculated the time lost by all this excessive commuting, “applying average travel speed for each metropolitan area to its benchmark commute distance, as opposed to its actual commute distance,” he explains in a blog post…

Hertz’s methodology may not be perfect. It might have served his analysis to have grouped these metros into narrower buckets, or by average commute distance rather than population. While it’s true that large cities tend to have longer commutes, there are exceptions. New Orleans and Louisville are non-dense, fairly sprawling cities, but their highways are built up enough that commute distances are fairly short. To really accurately assess the “sprawl tax” in cities like those, you’d have to include the other costs of spread-out development mentioned previously—the health impacts, the pollution, the car crashes, and so on. Hertz only addresses commute lengths and time.

In other words, a number of important conceptual decisions had to be made in order to arrive at this final figure. What might be more important in this situation is to know how different the final figure would be if certain calculations along the way were changed. Is it a relatively small shift or does this new methodology lead to figures much different than other studies? If they are really different, that doesn’t necessarily mean they are wrong but it might suggest more scrutiny for the methodology.

Another thought: it is difficult to put the $107 trillion into context. It is hard to understand really big numbers. Also, how does it compare to other activities? How much do Americans lose by watching TV? Or by using their smartphones? Or by eating meals? The number sounds impressive and is likely geared toward reducing sprawl but the figure doesn’t interpret itself.

11 recommendations from social scientists to journalists reporting scientific findings

Twenty social scientists were asked to give advice to journalists covering scientific research; here are a few of the recommendations.

1) Journalists often want clear answers to life and social problems. Individual studies rarely deliver that…

3) Journalists are obsessed with what’s new. But it’s better to focus on what’s old…

6) There’s a difference between real-world significance and statistical significance

10) Always direct readers back to the original research

And yes, not confusing correlation and causation is on the list. This would indeed be a good list for journalists and the media to keep in mind; the typical social science study produces pretty modest findings. Occasionally, there are studies that truly challenge existing theories and findings or these shifts might happen across a short amount of time or within a few studies.

At the same time, this would be a good list for the general public as well or starting students in a social science statistics or research methods course. For example, students sometimes equate using statistics or numbers with “proof” but that is not really what social science studies provide. Instead, studies tend to provide probabilities – people are more or less likely to have a future behavior or attitude (and this is covered specifically in #5 in the list). Or, we may have to explain in class how studies add up over time and lead to a consensus within a discipline rather than having a single study provide all the evidence (#s 1, 2, 3 on the list).

“Pollsters defend craft amid string of high-profile misses”

Researchers and polling organizations continue to defend their efforts:

Pollsters widely acknowledge the challenges and limitations taxing their craft. The universality of cellphones, the prevalence of the Internet and a growing reluctance among voters to respond to questions are “huge issues” confronting the field, said Ashley Koning, assistant director at Rutgers University’s Eagleton Center for Public Interest Polling…

“Not every poll,” Koning added, “is a poll worth reading.”

Scott Keeter, director of survey research at the Pew Research Center, agreed. Placing too much trust in early surveys, when few voters are paying close attention and the candidate pools are their largest, “is asking more of a poll than what it can really do.”…

Kathryn Bowman, a public opinion specialist at the American Enterprise Institute, also downplayed the importance of early primary polls, saying they have “very little predictive value at this stage of the campaign.” Still, she said, the blame is widespread, lamenting the rise of pollsters who prioritize close races to gain coverage, journalists too eager to cover those results and news consumers who flock to those types of stories.

Given the reliance on data in today’s world, particularly in political campaigns, polls are unlikely to go away. But, there will be likely be changes in the future that might include:

  1. More consumers of polls, the media and potential voters, learn what exactly polls are saying and what they are not. Since the media seems to love polls and horse races, I’m not sure much will change in that realm. But, we need great numeracy among Americans to sort through all of these numbers.
  2. Continued efforts to improve methodology when it is harder to reach people and obtain representative samples and predict who will be voting.
  3. A consolidation of efforts by researchers and poling organizations as (a) some are knocked out by a string of bad results or high-profile wrong predictions and (b) groups try to pool their resources (money, knowledge, data) to improve their accuracy. Or, perhaps (c) polling will just become a partisan effort as more objective observers realize their efforts won’t be used correctly (see #1 above).

Can religion not be fully studied with surveys or do we not use survey results well?

In a new book (which I have not read), sociologist Robert Wuthnow critiques the use of survey data to explain American religion:

Bad stats are easy targets, though. Setting these aside, it’s much more difficult to wage a sustained critique of polling. Enter Robert Wuthnow, a Princeton professor whose new book, Inventing American Religion, takes on the entire industry with the kind of telegraphed crankiness only academics can achieve. He argues that even gold-standard contemporary polling relies on flawed methodologies and biased questions. Polls about religion claim to show what Americans believe as a society, but actually, Wuthnow says, they say very little…

Even polling that wasn’t bought by evangelical Christians tended to focus on white, evangelical Protestants, Wuthnow writes. This trend continues today, especially in poll questions that treat the public practice of religion as separate from private belief. As the University of North Carolina professor Molly Worthen wrote in a 2012 column for The New York Times, “The very idea that it is possible to cordon off personal religious beliefs from a secular town square depends on Protestant assumptions about what counts as ‘religion,’ even if we now mask these sectarian foundations with labels like ‘Judeo-Christian.’”…

These standards are largely what Wuthnow’s book is concerned with: specifically, declining rates of responses to almost all polls; the short amount of time pollsters spend administering questionnaires; the racial and denominational biases embedded in the way most religion polls are framed; and the inundation of polls and polling information in public life. To him, there’s a lot more depth to be drawn from qualitative interviews than quantitative studies. “Talking to people at length in their own words, we learn that [religion] is quite personal and quite variable and rooted in the narratives of personal experience,” he said in an interview…

In interviews, people rarely frame their own religious experiences in terms of statistics and how they compare to trends around the country, Wuthnow said. They speak “more about the demarcations in their own personal biographies. It was something they were raised with, or something that affected who they married, or something that’s affecting how they’re raising their children.”

I suspect such critiques could be leveled at much of survey research: the questions can be simplistic, the askers of the questions can have a variety of motives and skills in developing useful survey questions, and the data gets bandied about in the media and public. Can surveys alone adequately address race, cultural values, politics views and behaviors, and more? That said, I’m sure there are specific issues with surveys regarding religion that should be addressed.

I wonder, though , if another important issue here is whether the public and the media know what to do with survey results. This book review suggests people take survey findings as gospel. They don’t know about the nuances of surveys or how to look at multiple survey questions or surveys that get at similar topics. Media reports on this data are often simplistic and lead with a “shocking” piece of information or some important trend (even if the data suggests continuity). While more social science projects on religion could benefit from mixed methods or by incorporating data from the other side (whether quantitative or qualitative), the public knows even less about these options or how to compare data. In other words, surveys always have issues but people are generally innumerate in knowing what to do with the findings.

The biggest time-use diary archive in the world

Numerous scholars are making use of the 850,000+ person days recorded in diaries and held in a UK archive:

Today, these files are part of the biggest collection of time-use diaries in the world, kept by the Centre for Time Use Research at the University of Oxford, UK. The centre’s holdings have been gathered from nearly 30 countries, span more than 50 years and cover some 850,000 person-days in total. They offer the most detailed portrait ever created of when people work, sleep, play and socialize — and of how those patterns have changed over time. “It certainly is unique,” says Ignace Glorieux, a sociologist at the Dutch-speaking Free University of Brussels. “It started quite modest, and now it’s a huge archive.”

The collection is helping to solve a slew of scientific and societal puzzles — not least, a paradox about modern life. There is a widespread perception in Western countries that life today is much busier than it once was, thanks to the unending demands of work, family, chores, smartphones and e-mails. But the diaries tell a different story: “We do not get indicators at all that people are more frantic,” says John Robinson, a sociologist who works with time-use diaries at the University of Maryland, College Park. In fact, when paid and unpaid work are totted up, the average number of hours worked every week has not changed much since the 1980s in most countries of the developed world…

But certain groups have experienced a different trend. According to analyses by Gershuny, Sullivan and other time-use researchers, two demographic groups are, in fact, working harder. One consists of employed, single parents, who put in exceptionally long hours compared to the average; the other comprises well-educated professionals, particularly those who also have small children. People in this latter group find themselves pushed to work hard and under societal pressure to spend quality time with their kids. “The combination of those pressures has meant that there is this group for which time pressure is particularly pertinent,” Sullivan says.

Some researchers are also testing new ways to record people’s activities as they can compare the results to the diaries:

In her preliminary analyses, Harms has found that gadget diaries and paper diaries show the same sequence of events, but that the gadgets reveal details that paper diaries missed. Most researchers in the field agree that the future lies in collecting data through phones and other devices. “Maybe this will bring a new boost to time-use research,” Glorieux says. He anticipates a situation in which reams of diary data — such as location, heart rate, calories burned and even ambient noise — are collected through phones and linked-up gadgets.

Much social science research is focused on particular events or aspects of people’s lives – not just a cross-section of time but also specific information measured in variables that we think might be related to other variables or that we think are worth measuring. In contrast, time-use diaries and other methods can help get at the mundane, everyday activity and interactions that make up a majority of our lives. Much of adult life is spent in necessary activities: making and eating food, resting and sleeping, cleaning, more passive leisure activities, caring for children. We also spend a decent amount of time alone or in our own head. These activities are occasionally punctuated by big events – something exciting happens at work or home, lively social interaction occurs, an important thought is had, etc. – to which we tend to pay more attention both in our own minds and in our data collection. Our methods should probably more closely match this regular activity and time-use diaries represent one way of doing this.

Competing population projections for Chicago

I highlighted one recent prediction that Chicago would soon trail Houston in population. Yet, another projection has Chicago gaining people and holding off Houston for longer. Which is right?

Data released by the Illinois Department of Health in February show that the population for Chicago, about 2.7 million in 2010, could decrease by 3 percent to 2.5 million by 2025. Meanwhile, Houston’s population could reach 2.54 million to 2.7 million in 2025, according to the Reuters report. But a recent population estimate by the Census Bureau shows an increase in population, rather than a decrease.

Census estimates released in June show that the population of Chicago increased by 1 percent from 2010 to 2014. So why is one projection showing a decrease, but another an increase?

Both data sets are based on estimates and assumptions, says Rob Paral, a Chicago-based demographer. Unlike the 2000 or 2010 census, where all residents answer a questionnaire, any interim projections or estimates must use sampling or a formula based on past population statistics to calculate population…

“Trend data do not support any increase in the projections for Chicago in the next 10 years,” said Bill Dart, the deputy director of policy, planning and statistics at the health department. Dart explained that the estimates from the census use a different formula than the health department. And factors such as births, deaths, migration, economic boons or natural disasters can disrupt projections.

Two groups dealing with population data that come to opposite conclusions. Two ways we might approach this:

  1. The differences are due to slightly different data, whether in the variables used or the projection models. We could have a debate about which model or variables are better for predicting population. Have these same kind of variables and models proven themselves in other cities? (Alternately, are there factors that both models leave out?)
  2. Perhaps the two predictions aren’t that different: one is suggesting a slight decline and one predicts a slight increase. Could both predictions be within the margin of error? We might be really worried if one saw a huge drop-off coming and the other disagreed but both projections here are not too different from no change at all. Sure, the media might be able to say the predictions disagree but statistically there is not much difference.

The answer will come in time. Still, projections like these still carry weight as they provide grist for the media, things for politicians to grab onto, and may just influence the actions of some (is Chicago or Houston a city on the rise?).