Here is a lot of interesting poll data collected during three years, April 2009 to early 2011, of the Great Recession. From a quick glance at the data, there is quite a bit of uncertainty and the country could go in a number of directions.
Tag Archives: surveys
A declining response to customer surveys?
Perhaps you, like me, has received an endless stream of invitations to take customer surveys on your receipts, in your email box, or while browsing a website. Experts note that the proliferation of these surveys may lead to a lower response rate and lower-quality data:
Surely, it’s nice to be courted for input, at least sometimes. But some consumers say they’re fed up with giving time-consuming feedback for free, don’t like being drawn into a data web used to evaluate employees or feel companies don’t act on the advice they get. Others say they simply don’t have anything revelatory to impart about, say, ordering a shirt or buying a package of pens…
“Survey fatigue” has long been a concern among pollsters. Some social scientists fear a pushback on feedback could hamper important government data-gathering, as for the census or unemployment statistics.
If more people say no to those, “the data, possibly, become less trustworthy,” said Judith Tanur, a retired Stony Brook University sociology professor specializing in survey methodology.
Response rates have been sinking fast in traditional public-opinion phone polls, including political ones, said Scott Keeter, the Pew Research Center’s survey director and the president of the American Association for Public Opinion Research. Pew’s response rates have fallen from about 36 percent in 1997 to 11 percent last year, he said. The rate includes households that weren’t reachable, as well as those that said no.
This is an issue that is bigger than customer surveys: it can be harder to reach people today with surveys because of call screening, the inability to contact people on cell phones, and the problems with doing web surveys. All of this means that people who conduct surveys will have to work even harder to get people to respond.
I wonder if the solution is to give customers better incentives for filling out surveys. A lot of these surveys include the chance for winning a prize but perhaps these could be increased or customers could earn points (and be able to redeem them) for giving consistent feedback.
I can honestly say that I very rarely fill out such surveys, even knowing how difficult it is for companies and research organizations to obtain such information. I recently started filling out a survey for Marriott after staying a few nights but the survey was ridiculously long and detailed so I quit 30% in.
Two issues with Most Admired poll: a large gap between #1 and others, low numbers for #1
While it is interesting to note that sitting presidents tend to lead in Gallup’s “Most Admired Lists,” two other things immediately struck me when looking at the tables:
1. There is a relatively big gap between #1 for most admired man and woman and everyone else. This year, President Obama is at 17% and his next closest competitor is at 3% while Hillary Clinton is also at 17% and her next competitor is at 7%. Since Gallup asks this as an open-ended question (exact phrasing: “What man that you have heard or read about, living today in any part of the world, do you admire most? And who is your second choice?”), it suggests that people name famous people, particularly types who are likely to be in the news a lot and whose positions are notable. If this is the case, is this really a survey about who is most admired or more about who is most well-known?
2. The leaders in each category are only at 17% and their competitors are quite a ways back. This could lead to several suggestions. Perhaps Americans don’t think in these terms much. For men, 32% said none or had no opinion and for women, 29% said none or had no opinion. Additionally, when asked about men 9% said a friend or relative and 12% said the same when asked about women. Even the current President is only most admired by 17%, suggesting that Americans are not necessarily looking to admire their political leaders. Another possible explanation might be that there is a wide range of admirable famous people in the United States. For men, the top 10 only account for 31% of responses though the top 10 females account for 47% of responses. This might reflect the lesser number of women in positions of power or leadership so more attention is focused on a select few.
This leads me to think that this poll may not really not tell us much about anything. Those selected as admired have relatively low figures, certain positions in society lead to being selected, and there are clear leaders but then also a mass of closely-admired figures.
UPDATE 12/28/11 10:11 PM – There seems to be similar variability in a recent poll that asked Americans which celebrity they most wanted to live next door. Also:
The majority of surveyed adults (42 percent) said they did not want to live next to any celebrities. “As a voyeuristic culture that breathlessly tracks every celebrity movement, it’s extremely surprising to see so many Americans saying they wouldn’t like to live next to any celebrity at all,” said Zillow Chief Marketing Officer Amy Bohutinsky. “In fact, more people opted out of a celebrity neighbor in 2012 than in any of the past years we’ve run this poll.”
Perhaps Americans are more tired of famous people this year?
Occupy Wall Street to move into foreclosures?
As Occupy Wall Street moves forward, here is one of the next steps is to move into foreclosures:
Occupy Wall Street has left the street and gone legit. They’ve rented office space in the Financial District and meet daily at a public atrium inside Deutche Bank.
“We’ve managed to, in basically two months, propel the issues of inequality and social justice to the top of our national discussion,” said one Occupier.
In various cities today there were marches on a variety of issues, but the movement plans new tactics. On Tuesday around the nation, it plans to occupy foreclosed homes. In mid-January, a call to pitch tents outside of Congress.
Foreclosures have generally taken a back seat recently to issues like jobs, stock markets, and Republican presidential nominees. Can OWS turn attention back to housing? It will be very interesting to see where they occupy homes (the worst areas like Merced, California or Las Vegas?), how they sustain their collective energy if they are more indeed spread out, and how neighbors respond.
Some recent polls on the most important issues in the minds of Americans:
–PwC Health Research Institute in mid-November: job creation is most important and healthcare and the deficit are tied at number two.
–Gallup in early November: the economy leads the way but there is no mention of housing or foreclosures.
–Rasmussen Reports in mid-October: economy leads by wide margin with 84% saying it is “very important.” No separate category for housing so hard to parse out jobs, stock market, housing.
Perhaps there aren’t many people tracking dissatisfaction with housing in recent months?
If these polls are correct, should the OWS focus on job creation and the economy rather than branch out into foreclosures/the housing market?
“What’s Your Problem?” misses an opportunity to explain survey research
The “What’s Your Problem?” column in the Chicago Tribune tackles the problems of consumers. Yesterday’s column involved a woman who had been called multiple times by a survey firm even after she asked to not be called again:
Over the following weeks, Scarborough representatives called Riedell repeatedly, asking her to participate in a 15-minute phone survey.
No matter how many times she refused their overtures, the calls kept coming.
Riedell said she asked each time to have her name taken off the call list but was told that representatives were not authorized to do so.
And so it continued through late summer and early fall. By the sixth call, Riedell decided she had heard enough. She emailed What’s Your Problem?
When contacted by the Tribune, the survey firm had this reponse which did not please Riedell:
Dercher said Riedell did not leave her name and phone number when she called Scarborough’s toll-free number, which are critical pieces of information so that the company can remove a respondent from the calling list.
Although her number could be randomly picked for another survey in the future, the odds are against that happening, Dercher said.
After reading Dercher’s email, Riedell said Scarborough’s response was, well, lame.
“The response says that their interviewers are not allowed to remove the name of a respondent from their calling list since the respondent’s name is confidential, but the interviewer already has the respondent’s name and phone number, otherwise they wouldn’t have been able to reach me by phone or address me by my name when I answered the phone,” Riedell said. “Sounds like gibberish to me.”
The column is clearly geared toward Riedell’s point of view and frankly, who likes to be called repeatedly by companies or survey organizations after refusing to participate? At the same time, let’s flip this around to see it from the opposite angle:
-Riedell was selected for the survey by random digit dialing. This is not unusual and telephone surveys are not covered by the Do Not Call registry.
-It doesn’t sound unusual that the survey interviewers didn’t have the power to remove her name from their lists. They were likely handed lists of numbers and told to call until they had an answer.
-Surveys often select their initial batch of respondents and then do whatever they can to get responses from them. The US Census Bureau goes to housing units repeated times in order to collect data because they want accurate data. (Of course, one Census worker who was doing his job last year was arrested for trespassing in Hawaii.) If survey companies simply gave up on people after one attempt, they would spend a lot more time and money and doing so might mess up their calibrated samples which are meant to represent larger populations.
In the end, Riedell may not like the system but in order to collect good data, survey companies may have to contact selected respondents multiple times. Since participation is voluntary, Riedell can opt out and perhaps Scarborough does need to have a more clearly delineated method by which people can opt out. Additionally, there may be some complications because Scarborough is a market survey research firm (tagline: Scarborough Research measures our shopping, media, and lifestyle behaviors) and are not academic researchers or political researchers (though push polls are very problematic). But this column could be much more informative about how survey research works and how consumers can respond to common requests for information rather than just suggesting that this woman should be able to more easily avoid telephone survey questions.
Americans don’t know about the level of wealth concentration in the United States
Sociologists have been talking about the growing levels of inequality in the United States for some time now. But a recent survey suggests that Americans are unaware just how much wealth is concentrated at the top (and there is a lot more information on the topic at this link):
A remarkable study (Norton & Ariely, 2010) reveals that Americans have no idea that the wealth distribution (defined for them in terms of “net worth”) is as concentrated as it is. When shown three pie charts representing possible wealth distributions, 90% or more of the 5,522 respondents — whatever their gender, age, income level, or party affiliation — thought that the American wealth distribution most resembled one in which the top 20% has about 60% of the wealth. In fact, of course, the top 20% control about 85% of the wealth (refer back to Table 1 and Figure 1 in this document for a more detailed breakdown of the numbers).
Even more striking, they did not come close on the amount of wealth held by the bottom 40% of the population. It’s a number I haven’t even mentioned so far, and it’s shocking: the lowest two quintiles hold just 0.3% of the wealth in the United States. Most people in the survey guessed the figure to be between 8% and 10%, and two dozen academic economists got it wrong too, by guessing about 2% — seven times too high. Those surveyed did have it about right for what the 20% in the middle have; it’s at the top and the bottom that they don’t have any idea of what’s going on.
Americans from all walks of life were also united in their vision of what the “ideal” wealth distribution would be, which may come as an even bigger surprise than their shared misinformation on the actual wealth distribution. They said that the ideal wealth distribution would be one in which the top 20% owned between 30 and 40 percent of the privately held wealth, which is a far cry from the 85 percent that the top 20% actually own. They also said that the bottom 40% — that’s 120 million Americans — should have between 25% and 30%, not the mere 8% to 10% they thought this group had, and far above the 0.3% they actually had. In fact, there’s no country in the world that has a wealth distribution close to what Americans think is ideal when it comes to fairness. So maybe Americans are much more egalitarian than most of them realize about each other, at least in principle and before the rat race begins.
So Americans have some ideas about what the wealth distribution should look like but not much of an idea of what it actually looks like. What exactly might they do if they knew the exact figures since it doesn’t seem to line up with what they think it should be?
Read about the possible effects of this heavy concentration of wealth, including helping to bring about our recent economic crisis, here.
How jobless Americans are spending their time
Some new research suggests that unemployed Americans are doing a variety of things:
One study last year found that much of the extra time gets spent sleeping and watching TV–leading to news reports that the jobless “frittered away” their time. Another analysis–this one released in January and co-written by Princeton economist Alan Krueger, who was announced Monday as the White House’s pick to serve as the chief economic adviser to President Obama–pointed in the same direction. It found that people tend to devote fewer hours to job searches the longer they’ve been unemployed, and that sleep–especially “sleep in the morning hours”–increases as joblessness goes on. Together, the studies appeared to create a picture of the unemployed as lazy and unproductive.But a sophisticated new analysis (pdf) complicates that picture. In a paper written for the National Bureau of Economic Research, Mark A. Aguiar, Erik Hurst, and Loukas Karabarbounis, using data from the American Time Use Survey, found that the jobless do spend about 30 percent of their extra time–the time they would otherwise have spent working–sleeping or watching TV, and another 20 percent on other leisure activities. But around 35 percent is spent doing unpaid but nonetheless important work, like child-care and housework. And other investments–things like education, health-care, and volunteer work –account for another 10 percent.
The notion advanced by some that jobless benefits are being used to support a life of leisure is, at best, simplistic.
But as Nancy Folbre, an economics professor at the University of Massachusetts, Amherst, notes, there’s a limit to how much useful unpaid work the jobless can do. “They lack the capital, land, tools and skills needed to flexibly shift from wage employment to production for their own use.,” she writes. “Even when they can make a partial shift, their productivity is likely to be lower in unpaid work than paid work.”
I’m a little surprised by the quote from an economist at the end: unpaid work still needs to be done by someone whether they currently have the skills for it or not. Perhaps she is referring to longer-term issues: do the unemployed go back to work (perhaps by changing fields or getting educated in new areas) or do they adjust to a life of unpaid work? In the meantime, there is a transition that has to be made. But I can imagine that some people would see this quote and wonder what this means for people who have always done unpaid work, particularly mothers.
Another way to interpret the earlier study that the unemployed enjoy a life of leisure is that this is due to feelings of restlessness and perhaps even depression.
In general, I find time use studies to be quite interesting. When you ask people general questions about how they spend their time, like how long they spend at work, the numbers can be quite inflated. The better studies require logs or diaries and ask questions about recent time periods where memories will not be as distorted. Here is how the American Time Use Study describes some of its methodology (starting at page 11 of this document):
The ATUS sample is randomized by day, with 50 percent of the sample reporting about
weekdays, Monday through Friday, and 50 percent reporting about Saturday and
Sunday. Designated persons must report about their activities on their designated day,
without any substitution of days…The ATUS interview is a combination of structured questions and conversational
interviewing. It consists of four major topics: the household roster, the time diary, the
summary questions, and a section related to information collected in the eighth CPS
interview. The portion of the interview relating to the CPS is divided into four sections:
labor force status, looking for work, industry and occupation, and earnings and school
enrollment. These questions are used to update or confirm time-sensitive CPS data or
to fill in missing CPS data. Each section is described below in more detail…For all parts of the interview except the collection of the time-use diary data (in
section 4, above), interviewers read scripted text on the CATI screen and enter the
reported responses.For the time-use diary, the interviewer uses conversational interviewing rather than
asking scripted questions. This is a more flexible interviewing technique designed to
allow the respondent to report on his or her activities comfortably and accurately. This
technique also allows interviewers to use methods to guide respondents through memory lapses, to probe in a nonleading way for the level of detail required to code activities, and to redirect respondents who are providing unnecessary information. As each activity is reported, the interviewer records the verbatim responses on a new activity line. The interviewers are trained to ensure that the respondent reports
activities (and activity durations) actually done on the previous (diary) day, not
activities done on a “usual” day. Interviewers do this by placing continual emphasis on
the word “yesterday” throughout the interview.
This study relies on both a diary and asking questions about yesterday.
Pew again asks for one-word survey responses regarding budget negotiations
I highlighted this survey technique in April but here it is again: Pew asked Americans to provide a one-word response to Congress’ debt negotiations.
Asked for single-word characterizations of the budget negotiations, the top words in the poll — conducted in the days before an apparent deal was struck — were “ridiculous,” “disgusting” and “stupid.” Overall, nearly three-quarters of Americans offered a negative word; just 2 percent had anything nice to say.
“Ridiculous” was the most frequently mentioned word among Democrats, Republicans and independents alike. It was also No. 1 in an April poll about the just-averted government shutdown. In the new poll, the top 27 words are negative ones, with “frustrating,” “poor,” “terrible,” “disappointing,“ “childish,” “messy” and “joke” rounding out the top 10.
And then we are presented a word cloud.
On the whole, I think this technique can suggest that Americans have generally unfavorable responses. But the reliance on particular terms is better for headlines than it is for collecting data. What would happen if public responses were split more evenly: what words/responses would then be used to summarize the data? The Washington Post headline (and Pew Research as well) can now use forceful and emotional words like “ridiculous” and “disgusting” rather than the more accurate numerical figures than about “three-quarters of Americans offered a negative word.” Why not also include an ordinal question (strongly disapprove to strongly approve) about American’s general opinion of debt negotiations in order to corroborate this open ended question?
This is a possibly interesting technique in order to take advantage of open ended questions without allowing respondents to give possibly lengthy responses. Open ended questions can produce a lot of data: there were over 330 responses in this survey alone. I’ll be interested to see if other organizations adopt this approach.
How Americans would respond to a new large religious building nearby
I’ll post a Quick Review of American Grace soon (see an earlier post here) but I wanted look at an excerpt about another topic I have written about recently: how suburban governments respond to requests for the construction of religious buildings (this includes churches and mosques). Here is a description of findings from the 2007 Faith Matters Survey (pages 512-514)
How Americans respond to land use matters involving religious groups depends on the religion in questions. According to the 2007 Faith Matter survey, an overwhelming majority of Americans (92 percent) say that the construction of a large Christian church in their community would either not both them (55 percent) or is something they would welcome (37 percent). This level of acceptance is high even among the most secular tenth of the population (87 percent), although their reaction is far less supportive. Eighty-two percent of the highly secular say that they would merely “not be bothered” by a large Christian church, while just 5 percent would explicitly welcome it.
Because of the near-ubiquity of Christian churches in American communities, we were also interested in reactions to a religious facility that would unfamiliar to many Americans, and so we asked about the construction of a “large Buddhist temple.”…
The point of asking about both kinds of religious structures it to distinguish among different reasons for opposing their construction. Some people might oppose both a large Christian church and a large Buddhist temple because they object to the construction of any sizable structure in their neighborhood, whether it be a church, a temple, a restaurant, a store. Or it could be because they have an aversion to religion of any kind. However, opposition to a Buddhist temple but not a Christian church would suggest that the concern lies with Buddhism specifically or perhaps “exotic” (or non-Christian) religions more generally.
For Buddhists who might be planning to build a temple, our results contain good news and bad news. The good news is the high overall support, at least in the abstract for a Buddhist temple. Three quarters of Americans (76 percent) say they have no problem with the construction of a large Buddhist temple in their neighborhood. The bad news is that only a small number (15 percent) would explicitly welcome it in their midst. Even worse news for the Buddhists is that one in five Americans (20 percent) say that they have no problem with a large Christian church but would object to a Buddhist temple…Approval of a Buddhist temple drops precipitously as personal religiosity increases…
These are interesting findings that suggest Americans are pretty favorable toward large new churches in their community and a majority would be favorable toward a large Buddhist temple. A few thoughts about these findings:
1. The interchanging of the term “community” and “neighborhood” bothers me. The original survey questions (see here) ask about buildings built in a community. I would assume many survey respondents would perceive a neighborhood as a smaller, closer geographic area and might respond differently. It would be one thing for a Naperville resident to express support for a Buddhist temple on the other side of the community, perhaps 7-8 miles away, compared to expressing support for a temple within a 15 minute walk.
2. I would suspect that more Americans would be less supportive if the questions asked about large religious buildings very close to their home. Residential neighbors often get worked up about such structures, not people from the other side of the community (unless it is a smaller community). This would be NIMBY in action.
3. The word “large” in the survey questions is a bit unclear here: are we talking about a megachurch or a congregation of 300? The sorts of problems Americans complain about regarding large structures, such as traffic, are larger with bigger buildings.
4. It’s too bad there isn’t a third question asking about responses to a proposal for a large mosque. While both Buddhists and Muslims are rated low according to larger American religious groups (see pages 501-509), I wonder if many Americans wouldn’t see Islam as more foreign than Buddhism.
On the whole, I am a bit skeptical that these survey results reflect zoning and municipal discussions regarding large religious congregations. Perhaps a very vocal minority tends to oppose such buildings – this tends to characterize a lot of local development discussions. But when residents feel threatened by such large structures, their magnanimity may decrease.
Poll figures on how the Rapture would have affected the Republican presidential field
Even as the news cycle winds down on Harold Camping and his prediction about the Rapture, Public Policy Polling (PPP) digs through some data to determine how the Rapture would have affected the field of Republican presidential candidates:
First off- no one really believed the Rapture was going to happen last weekend, or at least they won’t admit it. Just 2% of voters say they thought that was coming on Saturday to 98% who say they did not. It’s really close to impossible to ask a question on a poll that only 2% of people say yes to. A national poll we did in September 2009 found that 10% of voters thought Barack Obama was the Anti-Christ, or at least said they thought so. That 2% number is remarkably low.
11% of voters though think the Rapture will occur in their lifetimes, even if it didn’t happen last weekend. 66% think it will not happen and 23% are unsure. If the true believers who think the Rapture will happen in their lifetime are correct- and they’re the ones who had the strongest enough faith to get taken up into heaven- then that’s going to be worth a 2-5 point boost to Obama’s reelection prospects. That’s because while only 6% of independents and 10% of Democrats think the Rapture will happen during their lifetime, 16% of Republicans do. We always talk about demographic change helping Democrats with the rise of the Hispanic vote, but if the Rapture occurs it would be an even more immediate boost to Democratic electoral prospects.
Obama’s lead over Romney is 7 points with all voters, but if you take out the ones who think the Rapture will occur in their lifetime his advantage increases to 9 points. That’s because the Rapture voters support Romney by a 49-35 margin. Against Gingrich Obama’s 14 point lead overall becomes a 17 point one if you take out take the ‘Rapturers’ because they support Gingrich 50-37. And Obama’s 17 point lead over Palin becomes a 22 point spread without those voters because they support Palin 54-37.
Palin is the only person we tested on this poll who is actually popular with people who think the Rapture is going to happen. She has a 53/38 favorability with them, compared to 33/41 for Romney, 26/48 for Gingrich, and a 31/58 approval for Obama. Palin’s problem is that her favorability with everyone who doesn’t think the Rapture will happen is 27/66.
What a great way to combine two of the media’s recent fascinations. I would guess PPP put this poll together solely to take advantage of this news cycle. Should we conclude that Democrats should have wished the Rapture to actually happen to improve their political chances?
Of course, all of this data should be taken with a grain of salt as only 2% of the voters believed the Rapture was going to happen this past weekend and 11% believe it will happen in their lifetimes. These small numbers are out of a total sample of 600 people, meaning that about 12 people thought the Rapture would happen on Saturday and about 66 thought it would happen while they are alive. And this is all with a margin of error of plus or minus 4 percent, suggesting all of these numbers could be really, really small and not generalizable.
Do polls/surveys like these help contribute to giving all polls/surveys a bad reputation?
Legally Sociable 