Tag Archives: surveys

Pew using word frequencies to describe public’s opinion of budget negotiations

Posted on by

In the wake of the standoff over a federal government shutdown last week, Pew conducted a poll of Americans regarding their opinions on this event. One of the key pieces of data that Pew is reporting is a one-word opinion of the proceedings:

The public has an overwhelmingly negative reaction to the budget negotiations that narrowly avoided a government shutdown. A weekend survey by the Pew Research Center for the People & the Press and the Washington Post finds that “ridiculous” is the word used most frequently to describe the budget negotiations [29 respondents], followed by “disgusting,” [22 respondents] “frustrating,” [14 respondents] “messy,” [14 respondents] “disappointing” [13 respondents] and “stupid.” [13 respondents]

Overall, 69% of respondents use negative terms to describe the budget talks, while just 3% use positive words; 16% use neutral words to characterize their impressions of the negotiations. Large majorities of independents (74%), Democrats (69%) and Republicans (65%) offer negative terms to describe the negotiations.

The full survey was conducted April 7-10 among 1,004 adults; people were asked their impressions of the budget talks in interviews conducted April 9-10, following the April 8 agreement that averted a government shutdown.

I would be hesitant about leading off an article or headline (“Budget Negotiations in a Word – “Ridiculous”) with these word frequencies since they generally were used by few respondents: the most common response, “ridiculous,” was only given by 2.9% of the survey respondents (based on the figures here of 1,004 total respondents). I think the better figures to use would be the broader ones about negative responses where 69% used negative terms and a majority of all political stripes used a negative descriptor.

You also have to dig into the complete report for some more information. Here is the exact wording of the question:

PEW.2A If you had to use one single word to describe your impression of the budget negotiations in Washington, what would that one word be? [IF “DON’T KNOW” PROBE ONCE: It can be anything, just the first word that comes to mind…] [OPEN END: ENTER VERBATIM RESPONSE]

Additionally, the full report says that this descriptor question was only asked of 427 respondents on April 9-10 (so my above percentage should be altered: it should be 29/427 = 6.8%). So this is a smaller sample answering this particular question; how generalizable are the results? And the most common response to this question is the other category with 202 respondents. Presumably, the “others” are mostly negative since we are told 69% use negative terms. (As a side note, why not separate out the “don’t knows” and “refused”? There are 45 people in this category but these seem like different answers.)

One additional thought I have: at least this wasn’t put into a word cloud in order to display the data.

The case of the insistent American Community Survey employee

Posted on by

The decennial US Census has its employees try to contact a household six times (see a quick summary of their procedures here). But the Problem Solver in the Chicago Tribune presents a case where a Census employee working on the American Community Survey (ACS) irritated a Chicago couple:

The first few requests were tolerable. A Census Bureau worker would knock on John and Beverly Scott’s door and ask them to fill out an American Community Survey. The McKinley Park couple would politely decline.

But as the days passed, the visits became more frequent and the requests more urgent.

Some evenings, the doorbell would ring at dinnertime, then again at 10 p.m…

Scott said the requests had become so repetitive and annoying, the couple began pulling the old “out-of-candy-on-Halloween trick.”

“I work afternoons, and I’m not home,” Scott said. “My wife has to sit with the lights off because she doesn’t want to be bothered.”

Often, even that doesn’t work.

“They knock and knock and knock and ring and ring and ring,” Beverly Scott said. “Knocking longer is not going to make me answer the door, and it’s not going to help if we’re not here.”

The final straw, John Scott said, was when a Census Bureau employee told him he would be fined $2,000 if he did not fill out the 48-question survey.

When contacted by the Problem Solver, the regional ACS office said the couple would not be fined (though the government could do this) and they would stop trying to contact the couple (and they did stop).

Surveys that pick out samples that are representative often will work hard to contact the initial respondents. If they can’t make contact or get a response, then they move on to other people who might fit what they are looking for, adding time and resources that need to be spend for the project.

Interestingly, the couple in question also notes that although they filled out their decennial survey, they are not interested in filling out the American Community Survey because they see it as too intrusive:

But they’re not too keen on the American Community Survey, a more in-depth, ongoing questionnaire the Census Bureau conducts to compile information on area demographics, consumer patterns and economic issues.

In particular, the Scotts did not want to answer questions they found too personal, such as inquiries about their income, when they left for work and their health.

“The new questionnaire has gone way over the line,” Scott said. “We have told the representative that we are not going to answer private questions, but they continue to come to our door at all hours of the day and night.”

There were occasional reports of people who felt the same about the 2010 decennial census with some suggesting that a Census should only gather a head count and no other information. But in the future, the US Census has suggested that the ACS will play an increasingly important role as the government looks to collect more frequent data. As the story suggests, the ACS data is important for determining “the Consumer Price Index and how federal funding is allocated.” Rather than waiting every 10 years for a more comprehensive counting, the ACS provides more up-to-data that governments (from the federal to local level), researchers, and the public can utilize.

2010 Census director on suburbanization of minorities

Posted on by

Sociologist Robert M. Groves spoke earlier this week “at an Advertising Research Foundation event.” In his comments, Groves noted one of the major demographic trends in America: more minorities are now in the suburbs.

Of course, if Groves — with a Ph.D. in sociology and a long-time Michigan professor — were to put out a “for hire” sign for TV networks, a bidding war could heat up between Univision and Telemundo. The story of the 2010 Census, which could have been written in 2005 (or 1995, for that matter), is the boom in Hispanic America…

Last year left Groves well-armed with figures about the Hispanic population, such as the prevalence of those speaking Spanish at home and English elsewhere. And he has much to say about a dispersal trend in the Hispanic community, the departure from cities. In the Atlanta area, for example, the number of Hispanic residents spreading to the collar counties is soaring.

“The suburbanization of the minority population is a phenomenon over the past decade,” Groves said.

While the American suburbs have typically been seen as places where whites attempted to escape the city and minority populations (“white flight”), the number of minorities in the suburbs has been on the rise (read about this on a national scale here and in the Chicago region here and here).

The article goes on to consider how Groves might also be in demand as businesses look to utilize this kind of demographic knowledge:

Broadly, Groves has some cred if he were to become a network ambassador to Madison Avenue. At some level, he’s overseen a massive campaign — stretching from a Super Bowl spot to targeted marketing in 28 languages — as with the Census spent $300 million to $400 million in advertising last year.

As the Bureau sought to get more Americans to return their questionnaires, it figured that for every 1% increase it produced, that would save $85 million in the costs associated with knocking on doors later.

“The message got through and it changed behavior,” Groves said.

The director can also say he can manage a budget. The Bureau returned $1.6 billion to the government last year as it completed its work.

Before becoming director of the 2010 Census, Groves was well known in sociology for his work with surveys. This article suggests that he could parlay this Census experience plus his prior research into a lucrative corporate position.

What can 90% of Americans agree on?

Posted on by

The answer: not much. Pew Research has an article about the small number of issues in which 90% of Americans agree:

Yet there are some opinions that 90% of the public, or close to it, shares — including a belief that citizens have a duty to vote, an admiration for those who get rich through hard work, a strong sense of patriotism and a belief that society should give everyone an equal opportunity to succeed. Pew Research’s political values surveys have shown that these attitudes have remained remarkably consistent over time.

The proportion saying they are very patriotic has varied by just four percentage points (between 87% to 91%) across 13 surveys conducted over 22 years. Similarly, in May 1987, 90% agreed with the statement: “Our society should do what is necessary to make sure everyone has an equal opportunity to succeed.” This percentage has remained at about 90% ever since (87% in the most recent political values survey).

Interestingly, these cited figures are about foundational values in American culture. Exactly what some of these things mean could be up for debate: how should one express their “very patriotic” feelings? What exactly should it look like so that “everyone has an equal opportunity to succeed”? But as values, voting, patriotism, and meritocracy are quite powerful. (And it would also be interesting to see who doesn’t agree with these values.)

We could also ask why exactly 90% is a cutoff we should care about. Here is an explanation:

[R]eaching the 90% threshold is a rare occurrence in public opinion surveys. In part, this reflects the tendency of polling organizations to focus on current issues about which there are often considerable differences of opinion. Nonetheless, even on issues where one would expect to find near-total agreement, the public’s views are far from unanimous.

This is why Pew highlights a recent finding: “fully 90% of the public said that they were hearing mostly bad news about gas prices.”

It would be interesting to see more data on this to know just how rare 90% agreement is. How often might we expect to see this out of all survey responses? How different is the 90% occurrence compared to 80% or even 70%? Is this lack of 90% agreement unusual only for the United States or does this apply to other nations as well?

Sociologist immerses himself in dumpster diving

Posted on by

Sociologist Jeff Ferrell spends “a couple of hours each day” dumpster diving. Here is some of what he has discovered while dumpster diving:

“There is a stereotype that (dumpster diving) is done mostly by the homeless. Yes, many were. But they were generous and helpful to me and helped me survive. But they were only one group doing this.”

Ferrell said there are several categories of trash pickers. “The good old boys,” he says, are an ethnic mix of mostly older males who drive pickup trucks and scrounge for scrap metal.

Some, Ferrell said, are immigrants from Mexico and Central America who came looking for the American dream and were left with “scraps of the American dream.”

Another group Ferrell describes a “freegans,” people who came out of the vegan movement and consider eating thrown away food less harmful to the earth than “going to Walmart to eat a vegetarian sandwich.”

Alternative artists also make frequent dumpster dives, searching for scrap metal, broken glass fragments and other material.

Ferrell’s friend, Dan Phillips, builds low-income homes in Texas made entirely from salvaged items.

“These people are not lazy, ignorant and shiftless,” said Ferrell. “They are remarkably resourceful and smart.”

The last group of participants, said the professor, are those who shop retail and don’t have to dumpster dive to survive.

This is not an area that many Americans spend much time thinking about but texts like Ferrell’s Empire of Scrounge and the 2001 book, Rubbish!, written by two archaeologists working on the University of Arizona’s garbage project, shed some light on what happens to what we throw away.

It is also interesting to note that Ferrell seems to become quite involved in his research. The article also mentions his research time of five years as a graffiti writer (published in the book Crimes of Style).  In both of these instances, studying dumpster diving and graffiti, it would be near impossible to conduct a typical survey or even a broad range of interviews due to the more hidden and deviant nature of these activities. Additionally, this consistent insider perspective can provide much different information including insights into motivations and social hierarchies within these activities.

How to measure happiness (“prosperity”) across countries

Posted on by

Here is a topic just perfect for a Research Methods class discussion about conceptualization and operationalization: how to measure happiness across countries. Here is a quick summary of how the Legatum Institute measured this and found that Norway is the happiest country in the world:

With this in mind, five years ago researchers at the Legatum Institute, a London-based nonpartisan think tank, set out to rank the happiest countries in the world. But because “happy” carries too much of a touchy-feely connotation, they call it “prosperity.”

Legatum recently completed its 2010 Prosperity Index, which ranks 110 countries, covering 90% of the world’s population.

To build its index Legatum gathers upward of a dozen international surveys done by the likes of the Gallup polling group, the Heritage Foundation and the World Economic Forum. Each country is ranked on 89 variables sorted into eight subsections: economy, entrepreneurship, governance, education, health, safety, personal freedom and social capital.

The core conceit: Prosperity is complex; achieving it relies on a confluence of factors that build on each other in a virtuous circle.

Ultimately how happy you are depends on how happy you’ve been. If you’re already rich, like Scandinavia, then more freedom, security and health would add the most to happiness. For the likes of China and India (ranked 88th), it’s more a case of “show me the money.” What they want most of all? The opportunity to prove to themselves that money doesn’t buy happiness.

Some quick thoughts on this:

1. This is a lot of dimensions and indicators to consider: 89 measures, 8 subcategories.

2. The change from “happiness” to “prosperity” is an interesting one. Happiness is indeed a fuzzy term. But prosperity often refers to material wealth in terms of income or buying power. This prosperity defined more broadly: material wealth plus freedoms plus level of services plus social interactions. The Legatum Website suggests the Index is “the world’s only global assessment of wealth and well-being.”

3. I would be curious to know how comparable the data is across countries and across the organizations that form and ask these survey questions.

4. In this complexity, it is interesting to note that prosperity means different things to countries in different stages.

5. Even with all of these measures, which measures are used and how this Institute weights these particular factors would matter for the outcome. For example, the story at Forbes suggests that improving a nation’s entrepreneurial culture could make a big difference in these rankings. And the United States is ranked #1 in health care because “$5,500 a year in per-capita health spending has resulted in excellent vaccination rates, water quality and sanitation.” The Legatum Institute itself seems to put a big emphasis on business.

6. How come so many of these lists come from Forbes? Beyond the answer that Yahoo has a deal with Forbes for content, this is an interesting way to drive web traffic: top ten lists that catch people’s attention. How useful these sorts of lists are is debatable but they are often interesting and quickly summarize complex areas of life.

Pew finds that landline-only surveys are biased toward Republicans

Posted on by

Polling techniques have become more complicated in recent years with the introduction of cell phones. In the past, researchers could reasonably assume most US residents could be accessed through a landline. However, Pew now suggests there may be a political bias in surveys that only access people though landlines:

Across three Pew Research polls conducted in fall 2010 — conducted among 5,216 likely voters, including 1,712 interviewed on cell phones — the GOP held a lead that was on average 5.1 percentage points larger in the landline sample than in the combined landline and cell phone sample…

The difference in estimates produced by landline and dual frame samples is a consequence not only of the inclusion of the cell phone-only voters who are missed by landline surveys, but also of those with both landline and cell phones — so called dual users — who are reached by cell phone. Dual users reached on their cell phone differ demographically and attitudinally from those reached on their landline phone. They are younger, more likely to be black or Hispanic, less likely to be college graduates, less conservative and more Democratic in their vote preference than dual users reached by landline…

Cell phones pose a particular challenge for getting accurate estimates of young people’s vote preferences and related political opinions and behavior. Young people are difficult to reach by landline phone, both because many have no landline and because of their lifestyles. In Pew Research Center surveys this year about twice as many interviews with people younger than age 30 are conducted by cell phone than by landline, despite the fact that Pew Research samples include twice as many landlines as cell phones.

This seems to make sense: those who have cell phones and don’t have landlines are likely to be different than those who are reached by landlines.

A few questions that I have: does this issue exist in all phone surveys today (and this article suggests there was a sizable differences between landline people and cell phone people in five of six surveys)? Have other polling firms had similar findings? If Pew now has some ideas about the extent of this issue, is the proper long-term response to call more cell phones or to weight the results more toward cell phone users?

One possible response would be to include multiple methods for more surveys. This might include samples of landline respondents, cell phone respondents, and web respondents. While this is more costly and time-consuming, research firms could then triangulate results.

Considering “polls gone wild”

Posted on by

The Associated Press released a story yesterday with this headline: “Polls gone wild: Political gripes in Internet age.” It is an interesting read about the role polls have played in the 2010 election season and I have a few interpretations regarding the story.

1. The griping of politicians about polls does not often seem to be based on the methodology of the poll. Rather, I think the politicians are trying to curry favor with supporters and voters who are also suspicious of polls. I would guess many Americans are suspicious of polls because they think they can be manipulated (which is true) and then throw out all poll results (when there are methods that make the polls better or worse). Some of this could be dealt with by dealing with innumeracy and educating citizens about how good polls are done.

2. There is a claim that earlier polls affect later polls and elections and that overall, polls help determine election outcomes. Are there studies that prove this? Or is this just more smoke and mirrors from politicians?

3. If there are charges to be made about manipulation, it sounds like the political campaigns are manipulating the figures more than the reputable polling firms which are aiming to be statistically sound.

4. Stories like this remind me of the genius of RealClearPolitics.com where multiple polls about the same races are put side by side. If one doesn’t trust polls as much, just look at how polls compare over time. The more reputable companies show generally similar results over time. Basing news stories and campaign literature on just one poll may look silly in a few years with all of these companies producing numerous polls on almost a daily basis.

New survey of Chicago Catholics

Posted on by

On Wednesday, a new comprehensive report on Chicago Catholics will be released. The report is based on data from “524 Catholics in Cook and Lake counties” and the project was headed up by sociologist Andrew Greeley. Here are a few of the findings:

In addition, 78 percent of the respondents said Catholicism is either “extremely important” or “very important” in their lives.

Greeley wrote that the survey suggests “two separate Catholic identities — an imaginative, story-telling identity and a rules identity,” commonly referred to as “Cafeteria Catholics.” Those Catholics revere the sacraments and run in primarily Catholic circles, but they make their own choices on moral, religious and political issues.

“The only safe prediction seems to be that … there will be, whether the leadership likes it or not, varied forms of affiliation with a Church most of them still love,” Greeley wrote. “Not Cafeteria Catholics so much as Smorgasbord Catholics, a rich and diverse collection of ways to affirm one’s Catholicism.”

These findings about Catholics don’t seem too different than findings from other studies of American religion that show people of faith don’t follow all of the doctrines or practices of their faith traditions. Overall, people have some basic Christian beliefs or notions about spirituality but then a wide range of opinions on moral and practical issues.

A few questions about the methodology: 524 as a total N seems a little small plus I wonder why they went for Catholics in Cook and Lake Counties and not some of the other collar counties. This reminds me that on maps that show the religious plurality in each American county, DuPage County is coded Catholic. I’d also be curious to know whether Latino Catholics, of which there are a large number in the Chicago region, have different views than other Catholics.

On list of generous nations, US ranks 5th

Posted on by

Gallup has released “The World Giving Index 2010” and the United States is tied for fifth with Switzerland and behind Australia and New Zealand (tied for first) and Ireland and Canada (tied for third).

It looks like respondents were asked whether they did three things within the past month: gave money to an organization, volunteered for an organization, or helped someone they didn’t know.

Gallup suggests “the level of satisfaction or happiness of the population is emerging as the key driver for increasing the giving of money.” They also argue there could be “a positive cycle of giving” where happier people give to others who then are more likely to give.

I would be interested to know how much a country’s culture affects this. Are there certain societal traits that lead to more giving? Or are there certain economic and governmental structures that encourage more giving?