Tag Archives: telephone surveys

Growing troubles in surveying Americans

Posted on by

International difficulties in polling are also present in the United States with fewer responses to telephone queries:

With sample sizes often small, fluctuations in polling numbers can be caused by less than a handful of people. A new NBC News/Wall Street Journal national survey of the Republican race out this week, for instance, represents the preferences of only 230 likely GOP voters. Analysis of certain subgroups, like evangelicals, could be shaped by the response of a single voter.Shifting demographics are also playing a role. In the U.S., non-whites, who have historically voted at a lower rate than whites, are likely to comprise a majority of the population by mid-century. As their share of the electorate grows, so might their tendency to vote. No one knows by how much, making turnout estimates hard…

To save money, more polling is done using robocalls, Internet-based surveys, and other non-standard methods. Such alternatives may prove useful but they come with real risks. Robocalls, for example, are forbidden by law from dialing mobile phones. Online polling may oversample young people or Democratic Party voters. While such methods don’t necessarily produce inaccurate results, Franklin and others note, their newness makes it harder to predict reliability…

As response rates have declined, the need to rely on risky mathematical maneuvers has increased. To compensate for under-represented groups, like younger voters, some pollsters adjust their results to better reflect the population — or their assessment of who will vote. Different firms have different models that factor in things like voter age, education, income, and historical election data to make up for the all the voters they couldn’t query.

The telephone provided new means of communication in society but also helped make national mass surveys possible once a majority of Americans had them. Yet, even with cell phone adoption increasing to over 90% in 2013 and cell phones spreading as fast as any technology (comparable to the television in the early 1950s), the era of the telephone as an instrument for survey data may be coming to an end.

Three other thoughts:

  1. Another issue at this point of the election cycle: there are so many candidates involved that it is difficult to get good data on all of them.
  2. If the costs of telephone surveys keep going up, might we see more door-to-door surveys? Given the increase in contractor/part-time work, couldn’t polling organizations get good idea from all over the country?
  3. If polls aren’t quite as accurate as they might have bee in the past, does this mean election outcomes will be more exciting for the public? If so, would voter turnout increase?

“What’s Your Problem?” misses an opportunity to explain survey research

Posted on by

The “What’s Your Problem?” column in the Chicago Tribune tackles the problems of consumers. Yesterday’s column involved a woman who had been called multiple times by a survey firm even after she asked to not be called again:

Over the following weeks, Scarborough representatives called Riedell repeatedly, asking her to participate in a 15-minute phone survey.

No matter how many times she refused their overtures, the calls kept coming.

Riedell said she asked each time to have her name taken off the call list but was told that representatives were not authorized to do so.

And so it continued through late summer and early fall. By the sixth call, Riedell decided she had heard enough. She emailed What’s Your Problem?

When contacted by the Tribune, the survey firm had this reponse which did not please Riedell:

Dercher said Riedell did not leave her name and phone number when she called Scarborough’s toll-free number, which are critical pieces of information so that the company can remove a respondent from the calling list.

Although her number could be randomly picked for another survey in the future, the odds are against that happening, Dercher said.

After reading Dercher’s email, Riedell said Scarborough’s response was, well, lame.

“The response says that their interviewers are not allowed to remove the name of a respondent from their calling list since the respondent’s name is confidential, but the interviewer already has the respondent’s name and phone number, otherwise they wouldn’t have been able to reach me by phone or address me by my name when I answered the phone,” Riedell said. “Sounds like gibberish to me.”

The column is clearly geared toward Riedell’s point of view and frankly, who likes to be called repeatedly by companies or survey organizations after refusing to participate? At the same time, let’s flip this around to see it from the opposite angle:

-Riedell was selected for the survey by random digit dialing. This is not unusual and telephone surveys are not covered by the Do Not Call registry.

-It doesn’t sound unusual that the survey interviewers didn’t have the power to remove her name from their lists. They were likely handed lists of numbers and told to call until they had an answer.

-Surveys often select their initial batch of respondents and then do whatever they can to get responses from them. The US Census Bureau goes to housing units repeated times in order to collect data because they want accurate data. (Of course, one Census worker who was doing his job last year was arrested for trespassing in Hawaii.) If survey companies simply gave up on people after one attempt, they would spend a lot more time and money and doing so might mess up their calibrated samples which are meant to represent larger populations.

In the end, Riedell may not like the system but in order to collect good data, survey companies may have to contact selected respondents multiple times. Since participation is voluntary, Riedell can opt out and perhaps Scarborough does need to have a more clearly delineated method by which people can opt out. Additionally, there may be some complications because Scarborough is a market survey research firm (tagline: Scarborough Research measures our shopping, media, and lifestyle behaviors) and are not academic researchers or political researchers (though push polls are very problematic). But this column could be much more informative about how survey research works and how consumers can respond to common requests for information rather than just suggesting that this woman should be able to more easily avoid telephone survey questions.