Tag Archives: statistics

Three changes that come with “The Rise of Poll Quants”

Posted on by

Nate Silver isn’t the only one making election predictions based on poll data; there are now a number of “poll quants” who are using similar techniques.

So what exactly do these guys do? Basically, they take polls, aggregate the results, and make predictions. They each do it somewhat differently. Silver factors in state polls and national polls, along with other indicators, like monthly job numbers. Wang focuses on state polls exclusively. Linzer’s model looks at historical factors several months before the election but, as voting draws nearer, weights polls more heavily.

At the heart of all their models, though, are the state polls. That makes sense because, thanks to the Electoral College system, it’s the state outcomes that matter. It’s possible to win the national vote and still end up as the head of a cable-television channel rather than the leader of the free world. But also, as Wang explains, it’s easier for pollsters to find representative samples in a particular state. Figuring out which way Arizona or even Florida might go isn’t as tough as sizing up a country as big and diverse as the United States.”The race is so close that, at a national level, it’s easy to make a small error and be a little off,” Wang says. “So it’s easier to call states. They give us a sharper, more accurate picture.”

But the forecasters don’t just look at one state poll. While most news organizations trot out the latest, freshest poll and discuss it in isolation, these guys plug it into their models. One poll might be an outlier; a whole bunch of polls are likely to get closer to the truth. Or so the idea goes. Wang uses all the state polls, but gives more weight to those that survey likely voters, as opposed to those who are just registered to vote. Silver has his own special sauce that he doesn’t entirely divulge.

Both Wang and Linzer find it annoying that individual polls are hyped to make it seem as if the race is closer than it is, or to create the illusion that Romney and Obama are trading the lead from day to day. They’re not. According to the state polls, when taken together, the race has been fairly stable for weeks, and Obama has remained well ahead and, going into Election Day, is a strong favorite. “The best information comes from combining all the polls together,” says Linzer, who projects that Obama will get 326 electoral votes, well over the 270 required to win. “I want to give readers the right information, even if it’s more boring.”

While it may not seem likely, poll aggregation is a threat to the supremacy of the punditocracy. In the past week, you could sense that some high-profile media types were being made slightly uncomfortable by the bespectacled quants, with their confusing mathematical models and zippy computer programs. The New York Times columnist David Brooks said pollsters who offered projections were citizens of “sillyland.”

Three things strike me from reading these “poll quants” leading up to the election:

1. This is what is possible when data is widely available: these pundits use different methods for their models but it wouldn’t be possible without accessible data, consistent and regular polling (at the state and national level), and relatively easy to use statistical programs. In other words, could this scenario have taken place even 20 years ago?

2. It will be fascinating to watch how the media deals with these predictive models. Can they incorporate these predictions into their typical entertainment presentation? Will we have a new kind of pundit in the next few years? The article still noted the need for these quantitative pundits to have personality and style so it their results are not too dry for the larger public. Could we end up in a world where CNN has the exclusive rights to Silver’s model, Fox News has rights to another model, and so on?

3. All of this conversation about statistics, predictions, and modeling has the potential to really show where the American public and elite stands in terms of statistical knowledge. Can people understand the basics of these models? Do they simply blindly trust the models because they are “scientific proof” or do they automatically reject them because all numbers can be manipulated? Do some pundits know just enough to be dangerous and ask endless numbers of questions about the assumptions of different models? There is a lot of potential here to push quantitative literacy as a key part of living in the 21st century world. And it is only going to get more statistical as more organizations collect more data and new research and prediction opportunities arise.

Correlation and not causation: Redskins games predict results of presidential election

Posted on by

Big events like presidential elections tend to bring out some crazy data patterns. Here is my nomination for the oddest one of this election season: how the Washington Redskins do in their final game before the election predicts the presidential election.

Since 1940 — when the Redskins moved to D.C. — the team’s outcome in its final game before the presidential election has predicted which party would win the White House each time but once.

When the Redskins win their game before the election, the incumbent party wins the presidential vote. If the Redskins lose, the non-incumbent wins.

The only exception was in 2004, when Washington fell to Green Bay, but George W. Bush still went on to win the election over John Kerry.

This is simply a quirk of data: how the Redskins do should have little to no effect on voting in other states. This is exactly what correlation without causation is about; there may be a clear pattern ut it doesn’t necessarily mean the two related facts cause each other. There may be some spurious association here, some variable that predicts both outcomes, but even that is hard to imagine. Yet, the Redskins Rule has garnered a lot of attention in recent days. Why? A few possible reasons:

1. It connects two American obsessions: presidential elections and the NFL. A sidelight: both may involve a lot of betting.

2. So much reporting has been done on the 2012 elections that this adds a more whimsical and mysterious element.

3. Humans like to find patterns, even if these patterns don’t make much sense.

What’s next, an American octopus who can predict presidential elections?

Sociologist defends statistical predictions for elections and other important information

Posted on by

Political polling has come under a lot of recent fire but a sociologist defends these predictions and reminds us that we rely on many such predictions:

We rely on statistical models for many decisions every single day, including, crucially: weather, medicine, and pretty much any complex system in which there’s an element of uncertainty to the outcome. In fact, these are the same methods by which scientists could tell Hurricane Sandy was about to hit the United States many days in advance…

This isn’t wizardry, this is the sound science of complex systems. Uncertainty is an integral part of it. But that uncertainty shouldn’t suggest that we don’t know anything, that we’re completely in the dark, that everything’s a toss-up.

Polls tell you the likely outcome with some uncertainty and some sources of (both known and unknown) error. Statistical models take a bunch of factors and run lots of simulations of elections by varying those outcomes according to what we know (such as other polls, structural factors like the economy, what we know about turnout, demographics, etc.) and what we can reasonably infer about the range of uncertainty (given historical precedents and our logical models). These models then produce probability distributions…

Refusing to run statistical models simply because they produce probability distributions rather than absolute certainty is irresponsible. For many important issues (climate change!), statistical models are all we have and all we can have. We still need to take them seriously and act on them (well, if you care about life on Earth as we know it, blah, blah, blah).

A key point here: statistical models have uncertainty (we are making inferences about larger populations or systems from samples that we can collect) but that doesn’t necessarily mean they are flawed.

A second key point: because of what I stated above, we should expect that some statistical predictions will be wrong. But this is how science works: you tweak models, take in more information, perhaps change your data collection, perhaps use different methods of analysis, and hope to get better. While it may not be exciting, confirming what we don’t know does help us get to an outcome.

I’ve become more convinced in recent years that one of the reasons polls are not used effectively in reporting is that many in the media don’t know exactly how they work. Journalists need to be trained in how to read, interpret, and report on data. This could also be a time issue; how much time to those in the media have to pore over the details of research findings or do they simply have to scan for new findings? Scientists can pump out study after study but part of the dissemination of this information to the public requires a media who understands how scientific research and the scientific process work. This includes understanding how models are consistently refined, collecting the right data to answer the questions we want to answer, and looking at the accumulated scientific research rather than just grabbing the latest attention-getting finding.

An alternative to this idea about media statistical illiteracy is presented in the article: perhaps the media perhaps knows how polls work but likes a political horse race. This may also be true but there is a lot of reporting on statistics on data outside of political elections that also needs work.

How one woman helped make preventable injuries an American public health issue

Posted on by

The epidemiologist Susan P. Baker devoted her career to making preventable injuries a public health issue. Here is part of the story:

She embarked on an independent research project — a comparison of drivers who were not responsible for their fatal crashes with drivers who were — and in 1968 she sent Haddon a letter seeking federal financing for her study. He came through with $10,000 and continued to finance her research after he became president of the Insurance Institute for Highway Safety a year later…

Among Baker’s most important legacies is the widespread use of the infant car seat. By examining data from car crashes, she demonstrated that the passengers most likely to die were those younger than 6 months. They were killed at double the rate of 1-year-olds and triple the rate for ages 6 to 12. Why? Because babies rested in their mothers’ arms or laps, often in the front passenger seat, and because their still-fragile bodies were more susceptible to fatal injury than those of older children. Baker published her study in the journal Pediatrics in 1979, making headlines in newspapers across the country…

Around that time, Baker was one of the main authors of a report calling for the creation of a federal injury-prevention agency. Today the National Center for Injury Prevention and Control coordinates with state programs and underwrites research projects aimed at preventing injury, ranging from the intentional (rape, homicide, suicide) to the unintentional (falls, residential fires, drownings)…

Of course, Baker knows that we can’t make the world completely injury-proof. But her decades of research show how fairly simple preventive measures — fences around swimming pools, bike helmets, childproof caps on medicine containers — can save thousands of lives.

I couldn’t help thinking while reading this story that it demonstrates the interplay between science, culture, and government. The first paragraph of the article argues that in the 1960s that few people worried about preventable injuries but this has clearly changed since. Aiding this process was new scientific findings about injuries as well as presentable statistics that captured people’s attention. This reminds me of sociologist Joel Best’s explanation in Damned Lies and Statistics that the use of statistics emerged in the mid 1800s because reformers wanted to attach numbers and science to social problems they cared about. But for these numbers to matter and the science to be taken seriously, you need a culture as well as institutions that see science as a viable way of knowing about the world. Similarly, the numbers themselves are not enough to immediately lead to change; social problems such as automobile deaths go through a process by which the public becomes aware, a critical mass starts pressing the issue, and leaders respond by changing regulations. Is it a coincidence that these concerns about public health began to emerge in the 1960s at the same time of American ascendency in the scientific realm, the growth of the welfare state, the continued development of the mass media as well as mass consumption, and an era of more movements calling for human rights and governmental protections? Probably not.

h/t Instapundit

World population in 1804 = Facebook users today

Posted on by

Here is an interesting, if not misguided, comparison of how many people are now Facebook users:

One billion people. That’s how many active monthly users Facebook has accrued in the eight years of its existence, the company announced today.

It took the population of modern humans about 200,000 years to reach that number, a milestone that was hit, demographers believe, just over two centuries ago in 1804 (bearing in mind that population tabs, then and now, are not exactly precise). Since then, human population has just exploded, enabled and protected by advances in medicine, agriculture, and hygiene. In the past year, it is estimated that the human headcount hit 7 billion.

I think I know what this comparison is trying to do: show the remarkable speed at which Facebook has attracted users. I agree. It has been remarkable.

At the same time, this is comparing apples to oranges. Yes, they are both large numbers of people. But one number is tied to human development, birth rates, life expectancy, technological improvement, and so on. This number reminds us of the broader scope of human history which is longer and progress is relatively slow. Having seven billion people on earth requires a lot of resources, space, and creative energy to tackle everyday and long-term problems. On the other side, you have Facebook, an Internet site that has attracted lots of users. While some of these users may be mega-users, people who are constantly online updating their status, tagging photos, reading other people’s walls, it is still just an online program, a relatively small part of human existence.

Perhaps there would be better ways to make a comparison to Facebook’s user total:

1. Looking at adoption rates compared to other technologies. In other words, is Facebook’s growth something completely new, a sign of the digital world, or does its adoption rate compare more to other technologies? Comparisons can be made here.

2. What one billion people in the world do on a daily basis or how many other objects have such broad appeal. For example, this website suggests there are 5.6 billion cell phone users in the world. (Meaning: Facebook has many more users to attract.)

Genius and creativity = “a probabilistic function of quantity”

Posted on by

I was recently reading a Malcolm Gladwell article about the invention of the computer mouse and came across this statistical definition of genius and creativity:

The psychologist Dean Simonton argues that this fecundity is often at the heart of what distinguishes the truly gifted. The difference between Bach and his forgotten peers isn’t necessarily that he had a better ratio of hits to misses. The difference is that the mediocre might have a dozen ideas, while Bach, in his lifetime, created more than a thousand full-fledged musical compositions. A genius is a genius, Simonton maintains, because he can put together such a staggering number of insights, ideas, theories, random observations, and unexpected connections that he almost inevitably ends up with something great. “Quality,” Simonton writes, is “a probabilistic function of quantity.”

Simonton’s point is that there is nothing neat and efficient about creativity. “The more successes there are,” he says, “the more failures there are as well”—meaning that the person who had far more ideas than the rest of us will have far more bad ideas than the rest of us, too.

To put this in graph terms: as time increases, a creative person has an increasing number of ideas, a line with positive slope. Underneath this overall line of ideas is another positive line tracking the unsuccessful ideas and below that, increasing steadily but perhaps at a slower rate, is the line of successful ideas. In other words, the more overall ideas someone has, the more failures but also the more quality ideas.

The rest of the article is about creating the right structural environment to take advantage of ideas. Most groups and organizations won’t recognize all the best ideas but innovative organizations find ways to encourage and push the good ideas to the top. Indeed, the clincher at the end of the article is that Steve Jobs, supposedly one of the best innovators America has had in recent decades, missed some opportunities as well.

Sociologist: “one-year change in test results doesn’t make a trend”

Posted on by

A sociologist provides some insights into how firms “norm” test scores from year to year and what this means about how to interpret the results:

The most challenging part of this process, though, is trying to place this year’s test results on the same scale as last year’s results, so that a score of 650 on this year’s test represents the same level of performance as a score of 650 on last year’s test. It’s this process of equating the tests from one year to the next which allows us to judge whether scores this year went up, declined or stayed the same.But it’s not straightforward, because the test questions change from one year to the next, and even the format and content coverage of the test may change.

Different test companies even have different computer programs and statistical techniques to estimate a student’s score and, hence, the overall picture of how a student, school or state is performing. (Teachers too, but that’s a subject for another day.)

All of these variables – different test questions from year to year; variations in test length, difficulty and content coverage; and different statistical procedures to calculate the scores – introduce some uncertainty about what the “true” results are…

In testing, every year is like changing labs, in somewhat unpredictable ways, even if a state hires the same testing contractor from one year to the next. For this reason, I urge readers to not react too strongly to changes from last year to this year, or to consider them a referendum on whether a particular set of education policies – or worse, a particular initiative – is working.

One-year changes have many uncertainties built into them; if there’s a real positive trend, it will persist over a period of several years. Schooling is a long-term process, the collective and sustained work of students, teachers and administrators; and there are few “silver bullets” that can be counted on to elevate scores over the period of a single school year.

Overall, this piece gives us some important things to remember: one data point is hard to put into context. You can draw a trend line between two data points. Having more data points gives you a better indication of what is happening over time. However, just having statistics isn’t enough; we also need to consider the reliability and validity of the data. Politicians and administrators seem to like test scores because they offer concrete numbers which can help them point out progress or suggest that changes need to be made. Yet, just because these are numbers doesn’t mean that there isn’t a process that goes into them or that we need to understand exactly what the numbers involve.

Positive results for teaching statistics by computer

Posted on by

A recent study shows that students taking an online statistics course utilizing software from Carnegie Mellon do better than students who take a hybrid course with a classroom classroom:

The study, called “Interactive Learning Online at Public Universities,” involved students taking introductory statistics courses at six (unnamed) public universities. A total of 605 students were randomly assigned to take the course in a “hybrid” format: they met in person with their instructors for one hour a week; otherwise, they worked through lessons and exercises using an artificially intelligent learning platform developed by learning scientists at Carnegie Mellon University’s Open Learning Initiative.

Researchers compared these students against their peers in the traditional-format courses, for which students met with a live instructor for three hours per week, using several measuring sticks: whether they passed the course, their performance on a standardized test (the Comprehensive Assessment of Statistics), and the final exam for the course, which was the same for both sections of the course at each of the universities…

The robotic software did have disadvantages, the researchers found. For one, students found it duller than listening to a live instructor. Some felt as though they had learned less, even if they scored just as well on tests. Engaging students, such as professors might by sprinkling their lectures with personal anecdotes and entertaining asides, remains one area where humans have the upper hand.

But on straight teaching the machines were judged to be as effective, and more efficient, than their personality-having counterparts.

As someone who regularly teaches both Statistics and Social Research (a research methods course), these findings are intriguing. I understand the urge to curb costs while still providing a good education. However, I have three questions that perhaps go beyond these findings:

1. Are there any benefits for students from being in a classroom for three hours a week beyond learning outcomes? Is there a social dimension to the classroom setting that could enhance learning? For example, it is common for professors to have students work in groups or with each other, sometimes with the idea that being able to teach or effectively help another student will increase a student’s learning. Also, I wonder about learning becoming strictly an individualistic activity. Sure, there are ways to do this online (discussion boards, using Skype, etc.) but does this replicate the kind of discussions faculty and students can have in a classroom?

2. Are there any professors in the United States who might secretly welcome not having to teach statistics?

3. Is there a point in a discipline, like statistics, where the difficulty of the subject matter makes it more helpful to have a live instructor? This study looked at introductory stats courses but would the findings be the same if the courses covered more advanced topics that require more “intuition” and “art” than pure steps or facts?

h/t Instapundit

Route sociology majors can go: data analyst

Posted on by

I try to remind my students in Statistics and Social Research that there is a need in a lot of industries for people who can collect and analyze data. I was reminded of this when I saw an obituary about a sociologist who had gone on to become a well-known medical data analyst:

A professor in the Department of Health Services at the UCLA Fielding School of Public Health, [E. Richard] Brown founded the UCLA Center for Health Policy Research in 1994.

One of the center’s major activities has been the development of the California Health Interview Survey, the premier source of information about individual and household health status in California. It has served as a model for health surveys for other states.

Brown was the founder and principal investigator for the survey, which produced its first data from interviews with more than 55,000 California households in 2001. Information from the survey, which has been conducted every two years, has been used by policymakers, community advocates, researchers and others.

And working with important data can then lead to public policy options:

“The single thing that makes Rick stand out in this field is that he had an extraordinary capacity to use evidence about the public’s health and strategize and advocate to turn that evidence into the best policy and action,” said Dr. Linda Rosenstock, dean of the UCLA Fielding School of Public Health.

In 1990, Brown was co-author of California’s first single-payer healthcare legislation. He also co-wrote several other healthcare reform bills over the last two decades…

He also was a full-time senior consultant to President Clinton’s Task Force on National Health Care Reform and served as a senior health policy advisor for the Barack Obama for President Campaign — as well as serving as an advisor to U.S. Sens. Bob Kerrey, Paul Wellstone and Al Franken.

We need more people to collect useful data and then interpret what they mean. These days, the problem often is not a lack of information; rather, we need to know how to separate the good data from the bad and then be able to provide a useful interpretation. While some students may prefer to skip over the methodological sections of articles or books, understanding how to collect and analyze data can go a long way. Additionally, learning about these methods and data analysis can help one move toward a sociological view of the social world where personal anecdotes don’t matter as much as broad trends and looking at how social factors (variables) are related to each other.

Data guru Hans Rosling named to Time’s 100 most influential people

Posted on by

Hans Rosling’s talks are fascinating as he makes data and charts exciting and explanatory in his own enthusiastic manner. Named as one of the 100 most influential people by Time, Rosling is profiled by sociologist and MD Nicholas Christakis:

Hans Rosling trained in statistics and medicine and spent years on the front lines of public health in Africa. Yet his greatest impact has come from his stunning renderings of the numbers that characterize the human condition.

His 2006 TED talk, in which he animated statistics to tell the story of socio-economic development, has been viewed over 3.8 million times and translated into dozens of languages. His subsequent talks have moved millions of people worldwide to see themselves and our planet in new ways by showing how our actions affect our health and wealth and one another across space and time.

When you meet Rosling, 63, you are struck by his energy and clarity. He has the quiet assurance of a sword swallower (which he is) but also of a man who is in the vanguard of a critically important activity: advancing the public understanding of science.

What does Rosling make of his statistical analysis of worldwide trends? “I am not an optimist,” he says. “I’m a very serious possibilist. It’s a new category where we take emotion apart and we just work analytically with the world.” We can all, Rosling thinks, become healthy and wealthy. What a promising thought, so eloquently rendered with data.

Here are some of Rosling’s presentations that are well worth watching:

–200 Countries, 200 Years, 4 minutes – The Joy of Stats

–TED Talk: No More Boring Data

–TED Talk: The Good News of the Decade?

Here is what The Economist thinks are Rosling’s greatest hits.

I’ve used several of Rosling’s talk in class to illustrate what is possible with data and charts. Rosling gets at an important issue: data should tell a story and be interactive and available to people so they too can dig into it and understand the world better. By simply taking a chart and adding some extra information (like population size of a country displayed as a larger circle or being able to quickly show the quartile income distributions for a country) and the dimension of time, you can start to visualize patterns and possible explanations of how the world works.

(A side note: alas, I don’t think any sociologists were named as one of the 100 most influential people.)