97% response rate for American Community Survey

The Census Bureau regularly collects data through the American Community Survey and it has a high response rate:

“Since 2005, the American Community Survey has produced an annual overall survey response rate of around 97 percent,” says James Treat, chief of the American Community Survey Office. He compares filling out a survey to serving on a jury, paying taxes or getting a valid driver’s license.

The Census Bureau can do more than push patriotic buttons to persuade people. Under Title 13 of the U.S. Code, a person who willfully ignores the survey can be fined as much as $100. That fine could be as high as $500 if you lie — maybe claim to access the Internet through a “mobile broadband plan” because you don’t want to admit to having a “dial-up service.”

Treat says the Census Bureau has a thorough procedure to check for inconsistencies and inaccuracies and that people don’t need to worry about their private information being shared with immigration officials, cops, the IRS, employers or cable-service providers.

Given concerns today about survey fatigue, this response rate is astounding. It is a good thing since the data is used by all sorts of government agencies as well as researchers. Even though the ACS draws occasional attention from lawmakers who want to cut budgets, it also doesn’t rise the same kind of ire compared to the dicennial census and its massive undertaking.

Continuing political battles over Census data

Megan McArdle provides a reminder of the political nature of the Census:

If the Census is the key to political control, then you can expect parties to put more energy into gaming the census.  Arguably, you’re already seeing this: Republicans are now making their second attempt to defund the American Community Survey, which uses sampling to generate data between censuses.  The American Community Survey is not used for districting, but it is used for all manner of other policy purposes.

As the political fault lines harden in Congress, the battlegrounds are moving back to more hidden levers of policymaking.  There are the courts, of course: we’re now in the third decade of a mostly undeclared war to gain control of the Supreme Court and do some unelected legislating.  Data gathering and research funding are coming under fierce scrutiny.  And on the national security front, secrecy and executive orders seem to be the order of the day for whoever is in the White House.

Before you say it, no, this isn’t just Republicans.  But it’s not good on either side.  As the legislature has ceased being able to legislate, both parties almost have to resort to more undemocratic methods to achieve their goals.  The casualties, like judicial impartiality and good data for policymaking, are vastly more important than the causes for which this war is allegedly being fought.

To see more details of the recent Republican defunding attempt, see here.

Data is rarely impartial: the processes of by which it is collected, interpreted, and then used in policy can be quite political. That doesn’t mean that is has to be. Much of the grounding for social science is the idea that data can be more objectively collected and analyzed. Yet, within the realms of politics where data is often a means to victory, having a good handle on data can go a long way, as we saw in the 2012 presidential election or currently in debates among Republicans about how to handle voter data.

In the end, it will be fascinating to see how big data, from the Census to Facebook, does or does not become political. There are a couple of fault lines in this debate. First, there are people who will argue that having such data is in itself political and dangerous while the opposite side will argue that having such data is necessary to have more efficient and business government and business. This could be a debate between libertarians and others: should there even be big data in the first place? Second, there is a good number of people who like the idea of collecting and using big data but debate who should be able to benefit from the data. Can the data be used for political ends? If government should have its hands on big data, perhaps it is okay for businesses? Should individual consumers have more power or control over their contributions and participation in big data?

h/t Instapundit

Census Bureau moving to more online data collection to save money

The US Census Bureau is collecting more information online in order to cut costs:

The Census Bureau already has started offering an Internet option to the 250,000 households it selects every month at random for the American Community Survey. Since becoming available in January, more than half the responses have come in on a secure site that requires codes and PIN numbers.

The bureau expects to use the Internet — plus smart phones and other technologies yet to be invented — for the next decen­nial census, in 2020.

The increasing reliance on technology is designed to save money. The 2010 Census cost $96 per household, including the American Community Survey that has replaced the old long form. That cost has more than doubled in two decades, up from $70 in 2000 and $39 as recently as 1990…

The Census Bureau spent two years running preliminary experiments in how people responded to American Commu­nity Survey questions on the computer screen. Five rounds of ­testing involved tracking eye movements as people scanned a Web page looking for which answer they wanted to check.

The households selected for the survey still get their first contact the old-fashioned way, with a mailed letter telling them the questionnaire is on its way. Then they receive a letter telling them how to respond over the Internet. If they don’t use that option, they get a 28-page paper form a few weeks later.

It is too bad this may be motivated primarily by money. I would hope it would be motivated more by wanting to collect better data and boost response rates. However, I’m glad they seem to have done a good amount of testing. But, the article fails to address one of the biggest issues with web surveys: can this technique be used widely with different groups in the US population or does it work best with certain groups (usually younger, more Internet access)? All this is related to how much money can be saved: what percentage of mailed forms or household visits can be eliminated with new techniques? And I would be interested in hearing more about using smartphones. The Internet may be horribly outdated even today for a certain segment of the population. Imagine a Census 2020 app – used via Google Glass.

 

Washington D.C. the wealthiest city/metropolitan area in the country

According to 2011 American Community Survey data, Washington D.C. is the wealthiest metropolitan area in the country:

D.C. area residents have a median household income of $86,680, well above the national average of $50,502.

The large salaries may be attributable to the nearly 47 percent of workers who hold college degrees, making Washington one of the most highly educated areas in the country.

The list also shows more adults in the area were able to find employment during a down economic time. Just 5.8 percent of the workforce were unemployed in 2011.

Only 8.3 percent of Washington homes are living below the poverty line — the fifth lowest ranking in the country.

Here is some common traits of the wealthier cities in the United States:

The biggest factor in determining a city’s income, according to Alex Friedhoff, a Research Analyst at Brookings Institute’s Metropolitan Policy Program, is the underlying industries that employ the most residents, as well as the type of jobs. High-tech jobs, particularly those related to computers and information technology, tend to pay higher salaries and are more likely to be located in areas with affluent residents. On the other hand, most of the jobs in the lower-income metro areas tend to be in retail, service, agriculture and low-tech manufacturing.

A review of the employment characteristics of the different cities confirms this. Included among the richest cities are the information technology centers of Boston and Boulder, the finance hub of Bridgeport-Stamford, and the San Jose region, better known as Silicon Valley, home to some of the largest chipmakers and computer parts manufacturers in the world. Nationwide, 10.7% of workers are employed in professional, scientific, and management positions. Of the 10 wealthy metro regions, nine have a larger proportion of workers in that sector. In Boulder, 21.9% fall into that category.

In the poorest economies, there is a much higher proportion of low-end manufacturing and retail jobs. In the U.S. as a whole, 11.6% of workers are employed in retail. In the 10 poorest metro areas, eight exceed that number by a wide margin, including Hot Springs, Arkansas, where 17.3% of its workforce is employed in retail.

Based on these listed traits, perhaps we can make this conclusion: cities that have better adapted to the new information age economy based on innovation, computers, and highly educated workers are doing the best in terms of income. Places that haven’t been able to attract this kind of industry are playing from behind.

An important note about these stories: while headlines suggest this data is about cities, it is really about metropolitan regions. So when Washington D.C. is cited as the wealthiest city, this is not quite true; the region is the wealthiest. While some might that the city itself is necessary for the whole region to exist or thrive, a lot of this wealth plus many of the jobs are actually suburban. Don’t confuse the two though this often happens in the media.

The implications of discontinuing the American Community Survey

This didn’t exactly make the front page this week but a vote in the House of Representatives about the American Community Survey could have a big impact on how we understand the United States. Nate Berg explains:

So the Republican-led House of Representatives this week voted 232-190 to eliminate the American Community Survey, the annual survey of about 3 million randomly chosen U.S. households that’s like the Census only much more detailed. It collects demographic details such as what sort of fuel a household uses for heating, the cost of rent or mortgage payments, and what time residents leave home to go to work.

In a post on the U.S. Census Bureau’s website, Director Robert Groves says the bill “devastates the nation’s statistical information about the status of the economy and the larger society. Modern societies need current, detailed social and economic statistics. The U.S. is losing them.”

While the elimination of the ACS would take a slight nibble out of the roughly $3.8 trillion in government expenditures proposed in the 2013 federal budget, its negative impacts could be much greater – affecting the government’s ability to fund a wide variety of services and programs, from education to housing to transportation.

The issue is that the information collected in the ACS is used heavily by the federal government to figure out where it will spend a huge chunk of its money. In a 2010 report for the Brookings Institution, Andrew Reamer found that in the 2008 fiscal year, 184 federal domestic assistance programs used ACS-related datasets to help determine the distribution of more than $416 billion in federal funding. The bulk of that funding, more than 80 percent, went directly to fund Medicaid, highway infrastructure programs and affordable housing assistance. Reamer, now a research professor George Washington University’s Institute of Public Policy, also found that the federal government uses the ACS to distribute about $100 billion annually to states and communities for economic development, employment, education and training, commerce and other purposes. He says that should the ACS be eliminated, it would be very difficult to figure out how to distribute this money where it’s needed…

And it’s not just government money that would be wasted. Reamer says many businesses are increasingly reliant on the market data available within the ACS, and that without it they would have much less success picking locations where their businesses would have market demand. It would affect businesses throughout the country, “from mom-and-pops to Walmart.”

Some history might also be helpful here. The United States has carried out a dicennial census since 1790 but the American Community Survey began in the mid 1990s. There has been talk in recent years of replacing the expensive and complicated dicennial census with a beefed up American Community Survey. There would be several advantages: it wouldn’t cost as much plus the government (and the country) would have more consistent information rather than having to wait every ten years. In other words, our country is rapidly changing and we need consistent information that can tell us what is happening.

In my mind, as a researcher who consistently uses Census data, dropping the ACS would be a big loss. The government funding is important but even more important to me would be losing the more up-to-date information the ACS provides. Without this survey, we would likely have to rely on private data which is often restrictive and/or expensive. For example, I’ve used ACS data to track some housing issues but without this, I’m not sure where I could get similar data.

This is part of a larger issue of conservatives wanting to limit the reach of the Census Bureau. The argument often is that the Census is too intrusive, therefore invading the privacy of citizens (see this 2011 story about an insistent ACS worker), and the Constitution only provides for a dicennial census. I wonder if these arguments are red herrings: there is a long history of battling over Census counts and timing depending on which political party might benefit. For example, see Republican claims that inappropriate sampling techniques were used to correct undercounts for big cities, claims that the Census “imputes” races to people (so mark your race as American!), or efforts by New York City to ask for a recount in order to boost their 2010 population figures, which are tied to funding. In other words, the Census can turn into a political football even though its data is very important and it uses social science research techniques.

Changes in the Census Bureau’s data collection between 2000 and 2010

While certain parts of the dicennial Census have remained the same, a sociologist outlines some data collection changes between 2000 and 2010:

In 2000 the short form included only eight questions for the householder and six for every person living in the household. These were the same basic questions that would be asked in 2010 about sex, age, relationship to the householder, race and Hispanic status.

About one in six households received the “long form,” which included some 53 questions. In addition to the basic questions in every census form, this survey also contained questions on a whole panoply of characteristics, including housing, moving in the last five years, employment, detailed income sources, military service, disability status, ancestry, place of birth and education.

Until 2010, the long form and short form were distributed in tandem, but then the government split them. Beginning in 2005, the Census Bureau began to collect data for the American Community Survey, which is very similar to the old long form. That survey gets responses from about 2 million households and residents of group quarters (prison, dormitory, institution) every year, and tracks the same sort of data that was produced by the long form sample. The survey takes place all year and interviews, where necessary, are conducted by permanent staff — not the temporary workers who interview for decennial census.

The Census Bureau releases three sets of data each year from the community survey: the one-year, the three-year and the five-year files. The one-year file is released for areas of at least 65,000 in population, the three-year file for areas of at least 20,000 and the five-year file for all of the areas for which the long form was released (block-groups, tracts and higher). The census releases data only for certain locations with larger populations at one and three-year intervals. This is because, since the ACS is a sample, its reliability depends on sample size.

So on Dec. 14, 2010, the Census Bureau released the first five-year file from the American Community Survey, including all the data that used to be released from the census long form. This totaled 32,000 columns with 675,000 rows of data. These data are comparable to what was compiled from the 2000 Census and allow one to answer many questions at the neighborhood level, where data are needed beyond the few questions in the 2010 census.

In summary, two things have changed:

1. The Census has been moving toward more frequent data collection for a while. Some of the stuff that used to be asked every 10 years is now being asked more frequently, providing more up-to-date information.

2. Beveridge goes on to explain a couple of issues with the American Community Survey data, one good and one bad. On the bad side: although it may be more frequent, the sample size is not as large as a decennial census, meaning that we can’t be as certain about the results. On the good side: the Census will no longer have to rely on large teams of temporary staff every ten years but will have trained professionals collecting data year after year. In the coming years, it will be interesting to see how researchers treat these new figures.

Overall, this is a reminder that official figures don’t simply happen. They are the product of particular methodologies and definitions and when these protocols change, so can the data.

Additionally, the data is easily accessible (here are the 2010 figures). I’m sure it will continue to get easier to access and analyze the data. This is good for democracy as it is relatively easy for citizens to see the current status and changes in their nation. I wish more people could or would use this data frequently to better understand their surroundings.

The case of the insistent American Community Survey employee

The decennial US Census has its employees try to contact a household six times (see a quick summary of their procedures here). But the Problem Solver in the Chicago Tribune presents a case where a Census employee working on the American Community Survey (ACS) irritated a Chicago couple:

The first few requests were tolerable. A Census Bureau worker would knock on John and Beverly Scott’s door and ask them to fill out an American Community Survey. The McKinley Park couple would politely decline.

But as the days passed, the visits became more frequent and the requests more urgent.

Some evenings, the doorbell would ring at dinnertime, then again at 10 p.m…

Scott said the requests had become so repetitive and annoying, the couple began pulling the old “out-of-candy-on-Halloween trick.”

“I work afternoons, and I’m not home,” Scott said. “My wife has to sit with the lights off because she doesn’t want to be bothered.”

Often, even that doesn’t work.

“They knock and knock and knock and ring and ring and ring,” Beverly Scott said. “Knocking longer is not going to make me answer the door, and it’s not going to help if we’re not here.”

The final straw, John Scott said, was when a Census Bureau employee told him he would be fined $2,000 if he did not fill out the 48-question survey.

When contacted by the Problem Solver, the regional ACS office said the couple would not be fined (though the government could do this) and they would stop trying to contact the couple (and they did stop).

Surveys that pick out samples that are representative often will work hard to contact the initial respondents. If they can’t make contact or get a response, then they move on to other people who might fit what they are looking for, adding time and resources that need to be spend for the project.

Interestingly, the couple in question also notes that although they filled out their decennial survey, they are not interested in filling out the American Community Survey because they see it as too intrusive:

But they’re not too keen on the American Community Survey, a more in-depth, ongoing questionnaire the Census Bureau conducts to compile information on area demographics, consumer patterns and economic issues.

In particular, the Scotts did not want to answer questions they found too personal, such as inquiries about their income, when they left for work and their health.

“The new questionnaire has gone way over the line,” Scott said. “We have told the representative that we are not going to answer private questions, but they continue to come to our door at all hours of the day and night.”

There were occasional reports of people who felt the same about the 2010 decennial census with some suggesting that a Census should only gather a head count and no other information. But in the future, the US Census has suggested that the ACS will play an increasingly important role as the government looks to collect more frequent data. As the story suggests, the ACS data is important for determining “the Consumer Price Index and how federal funding is allocated.” Rather than waiting every 10 years for a more comprehensive counting, the ACS provides more up-to-data that governments (from the federal to local level), researchers, and the public can utilize.