Tag Archives: sample size

The Internet and social media mean our reference group is everyone and not just family and friends

Posted on by

Related to the post yesterday about the power of statistics on college campuses, here is a similar matter: how much do we compare our behavior today to “everyone” or “larger patterns” rather than just family and friends around us?

Photo by fauxels on Pexels.com

The connection is not just the Internet and social media and the way they connect us to more people and narratives. This is a change in statistics: we think we can see larger patterns and we can access more information.

Whether what we see on social media is a real pattern might not matter. (A reminder: relatively few people are active on Twitter.) We see more online and we can see what people are highlighting. This might appear as a pattern.

Not too long ago, we were more limited in our ability to compare our actions to others. The mass media existed but in more packaged forms (television, radio, music, films, newspapers, etc.) rather than the user-driven content of social media. The comparisons to that mass media still mattered – I remember sociologist Juliet Schor’s argument in The Overspent American of how increased TV watching was related to increased consumption – but people’s ties to their family and friends in geographic proximity were likely stronger. Or, in Robert Putnam’s Bowling Alone world, people spent a lot more time in local organizations and groups rather than in the broad realms of the Internet and social media.

Now, we can easily see how our choices or circumstances compare to others. Even odd situations we find ourselves in quickly be matched across a vast set of platforms for similarities and differences. Whether our tastes are mainstream or unusual, we can see how they stack up. If I am on college campus X on one side of the country, I can easily see what is happening on college campuses around the world.

Even as the Internet and social media is not fully representative of people and society, it does offer a sample regarding what other people are doing. We may care less about what the people directly near us are doing and we can quickly see what broader groups are doing. We can live our everyday lives with a statistical approach: look at the big N sample and adjust accordingly.

Growing troubles in surveying Americans

Posted on by

International difficulties in polling are also present in the United States with fewer responses to telephone queries:

With sample sizes often small, fluctuations in polling numbers can be caused by less than a handful of people. A new NBC News/Wall Street Journal national survey of the Republican race out this week, for instance, represents the preferences of only 230 likely GOP voters. Analysis of certain subgroups, like evangelicals, could be shaped by the response of a single voter.Shifting demographics are also playing a role. In the U.S., non-whites, who have historically voted at a lower rate than whites, are likely to comprise a majority of the population by mid-century. As their share of the electorate grows, so might their tendency to vote. No one knows by how much, making turnout estimates hard…

To save money, more polling is done using robocalls, Internet-based surveys, and other non-standard methods. Such alternatives may prove useful but they come with real risks. Robocalls, for example, are forbidden by law from dialing mobile phones. Online polling may oversample young people or Democratic Party voters. While such methods don’t necessarily produce inaccurate results, Franklin and others note, their newness makes it harder to predict reliability…

As response rates have declined, the need to rely on risky mathematical maneuvers has increased. To compensate for under-represented groups, like younger voters, some pollsters adjust their results to better reflect the population — or their assessment of who will vote. Different firms have different models that factor in things like voter age, education, income, and historical election data to make up for the all the voters they couldn’t query.

The telephone provided new means of communication in society but also helped make national mass surveys possible once a majority of Americans had them. Yet, even with cell phone adoption increasing to over 90% in 2013 and cell phones spreading as fast as any technology (comparable to the television in the early 1950s), the era of the telephone as an instrument for survey data may be coming to an end.

Three other thoughts:

  1. Another issue at this point of the election cycle: there are so many candidates involved that it is difficult to get good data on all of them.
  2. If the costs of telephone surveys keep going up, might we see more door-to-door surveys? Given the increase in contractor/part-time work, couldn’t polling organizations get good idea from all over the country?
  3. If polls aren’t quite as accurate as they might have bee in the past, does this mean election outcomes will be more exciting for the public? If so, would voter turnout increase?

Social scientists critique Facebook’s study claiming the news feed algorithm doesn’t lead to a filter bubble

Posted on by

Several social scientists have some concerns about Facebook’s recent findings that its news feed algorithm is less important than the choices of individual users in limiting what they see to what they already agree with:

But even that’s [sample size] not the biggest problem, Jurgenson and others say. The biggest issue is that the Facebook study pretends that individuals choosing to limit their exposure to different topics is a completely separate thing from the Facebook algorithm doing so. The study makes it seem like the two are disconnected and can be compared to each other on some kind of equal basis. But in reality, says Jurgenson, the latter exaggerates the former, because personal choices are what the algorithmic filtering is ultimately based on:“Individual users choosing news they agree with and Facebook’s algorithm providing what those individuals already agree with is not either-or but additive. That people seek that which they agree with is a pretty well-established social-psychological trend… what’s important is the finding that [the newsfeed] algorithm exacerbates and furthers this filter bubble.”

Sociologist and social-media expert Zeynep Tufekci points out in a post on Medium that trying to separate and compare these two things represents the worst “apples to oranges comparison I’ve seen recently,” since the two things that Facebook is pretending are unrelated have significant cumulative effects, and in fact are tied directly to each other. In other words, Facebook’s algorithmic filter magnifies the already human tendency to avoid news or opinions that we don’t agree with…

Christian Sandvig, an associate professor at the University of Michigan, calls the Facebook research the “not our fault” study, since it is clearly designed to absolve the social network of blame for people not being exposed to contrary news and opinion. In addition to the framing of the research — which tries to claim that being exposed to differing opinions isn’t necessarily a positive thing for society — the conclusion that user choice is the big problem just doesn’t ring true, says Sandvig (who has written a paper about the biased nature of Facebook’s algorithm).

Research on political echo chambers has grown in recent years and has included examinations of blogs and TV news channels. Is Facebook “bad” if it follows the pattern of reinforcing boundaries? While it may not be surprising if it does, I’m reminded of what I’ve read about Mark Zuckerberg’s intentions for what Facebook would do: bring people together in ways that wouldn’t happen otherwise. So, if Facebook itself has the goal of crossing traditional boundaries, which are usually limited by homophily (people choosing to associate with people largely like themselves) and protecting the in-group against out-group interlopers, then does this mean the company is not meeting its intended goals? I just took a user survey from them recently that didn’t include much about crossing boundaries and instead asked about things like having fun, being satisfied with the Facebook experience, and whether I was satisfied with the number of my friends.

Sort this out: poll of 39 economists suggests “30% chance of recession”

Posted on by

Polling economists about whether the country is headed for a recession does not seem to be the best way to make predictions:

The 39 economists polled Aug. 3-11 put the chance of another downturn at 30% — twice as high as three months ago, according to their median estimates. That means another shock to the fragile economy — such as more stock market declines or a worsening of the European debt crisis — could push the nation over the edge.

Yet even if the USA avoids a recession, as economists still expect, they see economic growth muddling along at about 2.5% the next year, down from 3.1% in April’s survey. The economy must grow well above 3% to significantly cut unemployment…

The gloomier forecast is a stunning reversal. Just weeks ago, economists were calling for a strong rebound in the second half of the year, based on falling gasoline prices giving consumers more to spend on other things and car sales taking off as auto supply disruptions after Japan’s earthquake faded. In fact, July retail sales showed their best gain in four months.

But that was before European debt woes spread, the government cut its growth estimates for the first half of 2011 to less than 1%, and Standard & Poor’s lowered the USA’s credit rating after the showdown over the debt ceiling.

Here is what I find strange about this:

1. The headline meant to grab our attention focuses on the 30% statistic. Is this a good or bad figure? It is less than 50% (meaning there are less equal odds) but it is also double the prediction of predictions three months ago. Based on a 3 in 10 chance of a recession, how would the country and individual change their actions?

2. This comes from a poll of 39 economists. One, this isn’t that many. Two, how do we know that these economists know what they are talking about? How successful have their predictions been in the past? I see the advantages of “crowd-sourcing,” consulting a number of estimates to get an aggregate figure, but the sample could be larger and we don’t know whether these economists will be right. (Even if they are not right, perhaps it gives us some indication about what “leading economists” think and this could matter as well).

3. How much of this is based on real data versus perceptions of the economy? The article suggests this is a “stunning reversal” of earlier predictions and then cites some data that seems to be worse. These figures don’t determine everything. I wonder what it would take for economists to predict a recession – which numbers would have to be worse and how bad would they have to get?

4. Will anyone ever come back and look at whether these economists got it right?

In the end, I’m not sure this really tells us anything. I suspect it is these sorts of statistics and headlines that push people to throw up their hands altogether about statistics.

Claim: “Facebook knows when you’ll break up”

Posted on by

There is an interesting chart going around that is based on Facebook data and claims to show when people are more prone to break-up. Here is a quick description of the chart:

British journalist and graphic designer David McCandless, who specializes in showcasing data in visual ways, compiled the chart. He showed off the graphic at a TED conference last July in Oxford, England.

In the talk, McCandless said he and a colleague scraped 10,000 Facebook status updates for the phrases “breakup” and “broken up.”

They found two big spikes on the calendar for breakups. The first was after Valentine’s Day — that holiday has a way of defining relationships, for better or worse — and in the weeks leading up to spring break. Maybe spring fever makes people restless, or maybe college students just don’t want to be tied down when they’re partying in Cancun.

Potentially interesting findings and it is an interesting way to present this data. But when you consider how the data was collected, perhaps it isn’t so great. A few thoughts on the subject:

1. The best way to figure this out would be to convince Facebook to let you have the data for relationship status changes.

2. Searching for the word “breakup” and “broken up” might catch some, or perhaps even many ended relationships, but not all. Does everyone include these words when talking about ending a relationship?

3. Are 10,000 status updates a representative sample of all Facebook statuses?

4. Is there a lag time involved in reporting these changes? Monday, for example is the most popular day for announcing break-ups, not necessarily for break-ups occurring on that day. Do people immediately run to Facebook to tell the world that they have ended a relationship?

5. Does everyone initially “register” and then “unregister” a relationship on Facebook anyway?

The more I think about it, it is a big claim to make that “Facebook knows when you are going to break up” based on this data mining exercise.