Tag Archives: data

Living Earth Simulator to model social world

Posted on by

Here is an interesting project, the Living Earth Simulator, that hopes to take a lot of data and come to conclusions about social life:

Described as a “knowledge collider,” and now with a pledge of one billion euros from the European Union, the Living Earth Simulator is a new big data and supercomputing project that will attempt to uncover the underlying sociological and psychological laws that underpin human civilization. In the same way that CERN’s Large Hadron Collider smashes together protons to see what happens, the Living Earth Simulator (LES) will gather knowledge from a Planetary Nervous System (PNS — yes, really) to try to predict societal fluctuations such as political unrest, economic bubbles, disease epidemics, and so on.

Orchestrated by FuturICT, which is basically a consortium of preeminent scientists, computer science centers around the world, and high-power computing (HPC) installations, the Living Earth Simulator hopes to correlate huge amounts of data — including real-time sources such as Twitter and web news — and extant, but separate approaches currently being used by other institutions, into a big melting pot of information. To put it into scientific terms, the LES will analyze techno-socio-economic-environmental (!) systems. From this, FuturICT hopes to reveal the tacit agreements and hidden laws that actually govern society, rather than the explicit, far-removed-from-reality bills and acts that lawmakers inexorably enact…

The timing of EU’s billion-euro grant is telling, too. As you probably know, the European Union is struggling to keep the plates spinning, and the LES, rather handily, will probably be the most accurate predictor of economic stability in the world. Beyond money, though, it is hoped that the LES and PNS can finally tell us why humans do things, like watch a specific TV show, buy a useless gadget, or start a revolution.

Looking at the larger picture, the Living Earth Simulator is really an admission that we know more about the physical universe than the social. We can predict with startling accuracy whether an asteroid will hit Earth, but we know scant little about how society might actually react to an extinction-level event. We plough billions of dollars into studying the effects and extent of climate change, but what if understood enough of the psychology and sociology behind human nature to actually change our behavior?

I don’t know about the prospects of such a project but if the BBC is reporting on it, perhaps it has a future.

A couple of statements in the description above intrigue me:

1. The simulator will help uncover “the tacit agreements and hidden laws that actually govern society.” Do most social scientists think this is possible if we only had enough data and the right simulator?

2. The comparison between the natural and social sciences is telling. The portrayal here is that the natural sciences have come a lot further in studying nature than social scientists in studying human behavior. Is this true? Is this a fair comparison – natural systems vs. social systems? How much “unknown knowledge” is really in each realm?

3. The coding for this project must be immense.

4. The article makes no mention of utilizing social scientists to help develop this project and analyze the data though the group behind it does have some social scientists on board.

Why cases of scientific fraud can affect everyone in sociology

Posted on by

The recent case of a Dutch social psychologist admitting to working with fraudulent data can lead some to paint social psychology or the broader discipline of sociology as problematic:

At the Weekly Standard, Andrew Ferguson looks at the “Chump Effect” that prompts reporters to write up dubious studies uncritically:

The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior.

And when the research reaches beyond the classroom, it becomes sillier still…

Described in this way, it does seem like there could be real journalistic interest in this study – as a human interest story like the three-legged rooster or the world’s largest rubber band collection. It just doesn’t have any value as a study of abstract truths about human behavior. The telling thing is that the dullest part of Stapel’s work – its ideologically motivated and false claims about sociology – got all the attention, while the spectacle of a lunatic digging up paving stones and giving apples to unlucky commuters at a trash-strewn train station was considered normal.

A good moment for reaction from a conservative perspective: two favorite whipping boys, liberal (and fraudulent!) social scientists plus journalists/the media (uncritical and biased!), can be tackled at once.

Seriously, though: the answer here is not to paint entire academic disciplines as problematic because of one case of fraud. Granted, some of the questions raised are good ones that social scientists themselves have raised recently: how much about human activity can you discover through relatively small sample tests of American undergraduates? But good science is not based on one study anyway. An interesting finding should be corroborated by similar studies done in different places at different times with different people. These multiple tests and observations help establish the reliability and validity of findings. This can be a slow process, another issue in a media landscape where new stories are needed all the time.

This reminds me of Joel Best’s recommendations regarding dealing with statistics. One common option is to simply trust all statistics. Numbers look authoritative, often come from experts, and they can be overwhelming. Just accepting them can be easy. At the other pole is the common option of saying that all statistics are simply interpretation and are manipulated so we can’t trust any of them. No numbers are trustworthy. Neither approaches are good options but they are relatively easy options. The better route to go when dealing with scientific studies is to have the basic skills necessary to understand whether they are good studies or not and how the process of science works. In this case, this would be a great time to call for better training among journalists about scientific studies so they can provide better interpretations for the public.

In the end, when one prominent social psychologist admits to massive fraud, the repercussions might be felt by others in the field for quite a while.

What is “The Big Data Boom”on the Internet good for?

Posted on by

The Internet is a giant source of ready-to-use data:

Today businesses can measure their activities and customer relationships with unprecedented precision. As a result, they are awash with data. This is particularly evident in the digital economy, where clickstream data give precisely targeted and real-time insights into consumer behavior…

Much of this information is generated for free, by computers, and sits unused, at least initially. A few years after installing a large enterprise resource planning system, it is common for companies to purchase a “business intelligence” module to try to make use of the flood of data that they now have on their operations. As Ron Kohavi at Microsoft memorably put it, objective, fine-grained data are replacing HiPPOs (Highest Paid Person’s Opinions) as the basis for decision-making at more and more companies.

The wealth of data also makes it easy to run experiments:

Consider two “born-digital” companies, Amazon and Google. A central part of Amazon’s research strategy is a program of “A-B” experiments where it develops two versions of its website and offers them to matched samples of customers. Using this method, Amazon might test a new recommendation engine for books, a new service feature, a different check-out process, or simply a different layout or design. Amazon sometimes gets sufficient data within just a few hours to see a statistically significant difference…

According to Google economist Hal Varian, his company is running on the order of 100-200 experiments on any given day, as they test new products and services, new algorithms and alternative designs. An iterative review process aggregates findings and frequently leads to further rounds of more targeted experimentation.

This sounds like a social scientist’s dream – if we could get our hands on the data.

My big question about all of this data is this: what should be done with it? This article, and others I’ve seen, have said that it will transform business. If this is just a way for businesses to become more knowledgeable, more efficient, and ultimately, more profitable, is this enough? Occasionally, we hear of things like discovering and/or tracking epidemics by looking at search queries or tools like the “mechanical turk” to crowdsource small but needed work. On the whole, does the data from the Internet advance human flourishing or concentrate some benefits in the hands of a few or even hinder flourishing? Does this data give us insights into health and medicine, international relations, and social interactions or does it primarily give entrepreneurs and established companies the chance to make more money? Are these questions that anyone really asks or cares about?

A new statistic to measure chemistry in basketball

Posted on by

Team chemistry is an elusive concept to measure but three “quantitative traders in the financial world” have developed a new statistic they say tackles the subject:

We introduce a novel Skills Plus Minus (“SPM”) framework to measure on-court chemistry in basketball. First, we evaluate each player’s offense and defense in the SPM framework based on three basic categories of skills: scoring, rebounding, and ball-handling. We then simulate games using the skill ratings of the ten players on the court. The results of the simulations measure the effectiveness of individual players as well as the 5-player lineup, so we can then calculate the synergies of each NBA team by comparing their 5-player lineup’s effectiveness to the “sum-of-the-parts.” We find that these synergies can be large and meaningful. Because skills have different synergies with other skills, our framework predicts that a player’s value is dependent on the other nine players on the court. Therefore, the desirability of a free agent depends on the players currently on the roster. Indeed, our framework is able to generate mutually beneficial trades between teams…

The research team pored over a ton of data, ran countless simulations and looked at how many points certain combinations of skills created…

One pattern that emerged was that “rare events” (like steals/defensive ball-handling) tended to produce positive synergies, while “common events” (like defensive rebounds) produce negative synergies. How come? Because increasing a team’s rebounding rate from 70 percent of defensive rebounds (which would be lousy) to, say, 75 percent (very good) represents only a 7 percent increase. But upping offensive rebounds, which aren’t nearly as common as defensive rebounds, from a rate of 30 percent to 35 percent represents a robust 17 percent gain…

Figuring out the component parts of what we know as chemistry or synergy is one of the next great frontiers of this movement. It’s not enough to put an exceptional distributor on the floor. To maximize that point guard’s gifts, a team must surround him with the right combination of players — and that combination might not always be the sexiest free agents on the market.

Sports has so much data to pore over that researchers could be occupied with for a long time.

This particular question is fascinating because one could get a lot of answers to why certain five player units are successful from different actors such as coaches, players, commentators, and fans. Players might be easier to assess (ha – look at all the issues with drafting) but looking at units requires sharp analytical skills and an overall view of a team.

Which team(s) will be the first to utilize this statistic and really build team units rather than cobble together a number of good players and then try to squeeze the best out of them? Certain players who might be considered “busts” may simply be in the wrong systems and be the “missing piece” for another team.

After case of fraud, researchers discuss others means of “misusing research data”

Posted on by

The news that a prominent Dutch social psychologist published fraudulent work has pushed other researchers to talk about other forms of “misusing research data”:

Even before the Stapel case broke, a flurry of articles had begun appearing this fall that pointed to supposed systemic flaws in the way psychologists handle data. But one methodological expert, Eric-Jan Wagenmakers, of the University of Amsterdam, added a sociological twist to the statistical debate: Psychology, he argued in a recent blog post and an interview, has become addicted to surprising, counterintuitive findings that catch the news media’s eye, and that trend is warping the field…

In September, in comments quoted by the statistician Andrew Gelman on his blog, Mr. Wagenmakers wrote: “The field of social psychology has become very competitive, and high-impact publications are only possible for results that are really surprising. Unfortunately, most surprising hypotheses are wrong. That is, unless you test them against data you’ve created yourself.”…

To show just how easy it is to get a nonsensical but “statistically significant” result, three scholars, in an article in November’s Psychological Science titled “False-Positive Psychology,” first showed that listening to a children’s song made test subjects feel older. Nothing too controversial there.

Then they “demonstrated” that listening to the Beatles’ “When I’m 64” made the test subjects literally younger, relative to when they listened to a control song. Crucially, the study followed all the rules for reporting on an experimental study. What the researchers omitted, as they went on to explain in the rest of the paper, was just how many variables they poked and prodded before sheer chance threw up a headline-making result—a clearly false headline-making result.

If the pressure is great to publish (and it certainly is), then there have to be some countermeasures to limit unethical research practices. Here are a few ideas:

1. Giving more people access to the data. In this way, people could check up on other people’s published findings. But if the fraudulent studies are already published, perhaps this is too late.

2. Having more people have oversight over the project along the way. This doesn’t necessarily have to be a bureaucratic board but only having one researcher looking at the data and doing the analysis (such as in the Stapel case) means that there is more opportunity for an individual to twist the data. This could be an argument for collaborative data.

3. Could there be more space within disciplines and journals to discuss the research project? While papers tend to have very formal hypotheses, there is a lot of messy work that goes into these but very little room to discuss how the researchers arrived at them.

4. Decrease the value of media attention. I don’t know how to deal with this one. What researcher doesn’t want to have more people read their research?

5. Have a better educated media so that they don’t report so many inconsequential and shocking studies. We need more people like Malcolm Gladwell who look at a broad swath of research and summarize it rather than dozens of reports grabbing onto small studies. This is the classic issue with nutrition reporting: eggs are great! A new study says they are terrible! A third says they are great for pregnant women and no one else! We rarely get overviews of this research or real questions about the value of all this research. We just get: “a study proved this oddity today…”

6. Resist data mining. Atheoretical correlations don’t help much. Let theories guide statistical models.

7. Have more space to publish negative findings. This would help researchers feel less pressure to come up with positive results.

One of the new research frontiers: studying dating online

Posted on by

There are now a number of academics studying online dating sites as they allow insights into relationship formation that are difficult to observe elsewhere in large numbers:

Like contemporary Margaret Meads, these scholars have gathered data from dating sites like Match.com, OkCupid and Yahoo! Personals to study attraction, trust, deception — even the role of race and politics in prospective romance…

“There is relatively little data on dating, and most of what was out there in the literature about mate selection and relationship formation is based on U.S. Census data,” said Gerald A. Mendelsohn, a professor in the psychology department at the University of California, Berkeley…

Andrew T. Fiore, a data scientist at Facebook and a former visiting assistant professor at Michigan State University, said that unlike laboratory studies, “online dating provides an ecologically valid or true-to-life context for examining the risks, uncertainties and rewards of initiating real relationships with real people at an unprecedented scale.”…

Of the romantic partnerships formed in the United States between 2007 and 2009, 21 percent of heterosexual couples and 61 percent of same-sex couples met online, according to a study by Michael J. Rosenfeld, an associate professor of sociology at Stanford. (Scholars said that most studies using online dating data are about heterosexuals, because they make up more of the population.)

The rest of the article has some research findings about appearance, race, and political ideology derived from studies of online dating site members.

Researchers will go wherever the research subjects are so if the people are expanding their dating pools online, that is where the research to go. It would be interesting to hear if any of these researchers have received pushback from people within their own fields who scoff at online dating sites or ask them to demonstrate the worthiness of studying online behavior.

 

Predicting and preventing burglaries though statistical models in Indio, California

Posted on by

In January 2011, I wrote about how Santa Clara, California was going to use statistical models to predict where crime would take place and then deploy police accordingly. Another California community, Indio, is going down a similar route to reduce burglaries:

The Indio Police Department with the help of a college professor and a wealth of data and analysis is working on just that — predicting where certain burglaries will occur.The goal is to stop them from happening through effective deployment or preventative measures…

The police department began the Smart Policing Initiative a year ago with $220,617 in federal funding from the U.S. Department of Justice…

Robert Nash Parker, a professor of sociology at the University of California, Riverside and an expert on crime, is working with Indio.

On Friday, he shared his methodology for tracking truancy and burglary rates.

He used data from the police department, school district, U.S Census Bureau and probation departments, to create a model that can be used to predict such daytime burglaries.

Nash said that based on the data, truancy seems to lead to burglary hot spots.

A few issues come to mind:

1. Could criminals simply change up their patterns once they know about this program?

2. Do approaches like this simply treat the symptoms rather than the larger issues, in this case, truancy? It is a good thing to prevent crimes or arrest people quickly but what about working to limit the potential for crime in the first place?

3. I wonder how much data is required for this to work and how responsive it is to changes in the data.

4. Since this is being funded by a federal agency, can we expect larger roll-outs in the future? Think of this approach versus that of a big city like Chicago where there has been a greater emphasis on the use of cameras.

Changes in the Census Bureau’s data collection between 2000 and 2010

Posted on by

While certain parts of the dicennial Census have remained the same, a sociologist outlines some data collection changes between 2000 and 2010:

In 2000 the short form included only eight questions for the householder and six for every person living in the household. These were the same basic questions that would be asked in 2010 about sex, age, relationship to the householder, race and Hispanic status.

About one in six households received the “long form,” which included some 53 questions. In addition to the basic questions in every census form, this survey also contained questions on a whole panoply of characteristics, including housing, moving in the last five years, employment, detailed income sources, military service, disability status, ancestry, place of birth and education.

Until 2010, the long form and short form were distributed in tandem, but then the government split them. Beginning in 2005, the Census Bureau began to collect data for the American Community Survey, which is very similar to the old long form. That survey gets responses from about 2 million households and residents of group quarters (prison, dormitory, institution) every year, and tracks the same sort of data that was produced by the long form sample. The survey takes place all year and interviews, where necessary, are conducted by permanent staff — not the temporary workers who interview for decennial census.

The Census Bureau releases three sets of data each year from the community survey: the one-year, the three-year and the five-year files. The one-year file is released for areas of at least 65,000 in population, the three-year file for areas of at least 20,000 and the five-year file for all of the areas for which the long form was released (block-groups, tracts and higher). The census releases data only for certain locations with larger populations at one and three-year intervals. This is because, since the ACS is a sample, its reliability depends on sample size.

So on Dec. 14, 2010, the Census Bureau released the first five-year file from the American Community Survey, including all the data that used to be released from the census long form. This totaled 32,000 columns with 675,000 rows of data. These data are comparable to what was compiled from the 2000 Census and allow one to answer many questions at the neighborhood level, where data are needed beyond the few questions in the 2010 census.

In summary, two things have changed:

1. The Census has been moving toward more frequent data collection for a while. Some of the stuff that used to be asked every 10 years is now being asked more frequently, providing more up-to-date information.

2. Beveridge goes on to explain a couple of issues with the American Community Survey data, one good and one bad. On the bad side: although it may be more frequent, the sample size is not as large as a decennial census, meaning that we can’t be as certain about the results. On the good side: the Census will no longer have to rely on large teams of temporary staff every ten years but will have trained professionals collecting data year after year. In the coming years, it will be interesting to see how researchers treat these new figures.

Overall, this is a reminder that official figures don’t simply happen. They are the product of particular methodologies and definitions and when these protocols change, so can the data.

Additionally, the data is easily accessible (here are the 2010 figures). I’m sure it will continue to get easier to access and analyze the data. This is good for democracy as it is relatively easy for citizens to see the current status and changes in their nation. I wish more people could or would use this data frequently to better understand their surroundings.

Evangelicals and their propensity to think that everyone is against them

Posted on by

Sociologist Bradley Wright draws attention to an issue among evangelicals: a common belief that fellow Americans do not like them:

Similarly, somewhere along the line we evangelical Christians have gotten it into our heads that our neighbors, peers, and most Americans don’t like us, and that they like us less every year. I’ve heard this idea stated in sermons and everyday conversation; I’ve read it in books and articles.

There’s a problem, though. It doesn’t appear to be true. Social scientists have repeatedly surveyed views of various religions and movements, and Americans consistently hold evangelical Christians in reasonably high regard. Furthermore, social science research indicates that it’s almost certain that our erroneous belief that others dislike us is actually harming our faith.

The statistics Wright presents suggests evangelicals are somewhere in the middle of favorability among different religious groups. For example, a 2008 Gallup survey suggests Methodists, Jews, Baptists, and Catholics are viewed more favorably than evangelicals while Fundamentalists, Mormons, Muslims, Atheists, and Scientologists are viewed less favorably.

Wright goes on to argue (as he also does in this book) that the perceptions evangelicals have might be harmful:

If American evangelicals do have an image problem, it’s not our neighbors’ image of us; it’s our image of them. The 2007 Pew Forum study found that American Christians hold more negative views of “atheists” than non-Christians do of evangelical Christians. (The most recent Pew survey found similar attitudes; see the chart above.) Now, I am not a theologian, but this seems to be a problem. We Christians are called to love people, and as I understand it, this includes loving people who believe differently than we do. I’m not sure how we can love atheists if we don’t like them.

Ultimately, evangelical Christians might do well not to spend too much time worrying about what others think of us. Christians in general, and evangelical Christians in particular (depending on how you ask the question), are well-regarded in this country. If nothing else, there’s little we can do to change other people’s opinions anyway. Telling ourselves over and over that others don’t like us is not only inaccurate, it also potentially hinders the very faith that we seek to advance.

This is an ongoing issue with several aspects:

1. There is a disconnect between the numbers and the perceptions. Wright looks like he is trying to make a prolonged effort to bring these statistics to the masses. Will this data make a difference in the long run? How many evangelicals will ever hear about these statistics?

2. There may be positive or functional aspects to continually holding the idea that others don’t like you. Subgroups can use this idea to enhance solidarity and prompt action among adherents. Of course, these alarmist tendencies might not be helpful in the long run. (See a better explanation of this perspective from Christian Smith here.)

In the end, this is useful data but there is more that could be done to explain how these perceptions are helpful or not and what could or should be done to move in a different direction. Providing people with the right data and good interpretations is a good start but then people will want to know what to do next.

Pew again asks for one-word survey responses regarding budget negotiations

Posted on by

I highlighted this survey technique in April but here it is again: Pew asked Americans to provide a one-word response to Congress’ debt negotiations.

Asked for single-word characterizations of the budget negotiations, the top words in the poll — conducted in the days before an apparent deal was struck — were “ridiculous,” “disgusting” and “stupid.” Overall, nearly three-quarters of Americans offered a negative word; just 2 percent had anything nice to say.

“Ridiculous” was the most frequently mentioned word among Democrats, Republicans and independents alike. It was also No. 1 in an April poll about the just-averted government shutdown. In the new poll, the top 27 words are negative ones, with “frustrating,” “poor,” “terrible,” “disappointing,“ “childish,” “messy” and “joke” rounding out the top 10.

And then we are presented a word cloud.

On the whole, I think this technique can suggest that Americans have generally unfavorable responses. But the reliance on particular terms is better for headlines than it is for collecting data. What would happen if public responses were split more evenly: what words/responses would then be used to summarize the data? The Washington Post headline (and Pew Research as well) can now use forceful and emotional words like “ridiculous” and “disgusting” rather than the more accurate numerical figures than about “three-quarters of Americans offered a negative word.” Why not also include an ordinal question (strongly disapprove to strongly approve) about American’s general opinion of debt negotiations in order to corroborate this open ended question?

This is a possibly interesting technique in order to take advantage of open ended questions without allowing respondents to give possibly lengthy responses. Open ended questions can produce a lot of data: there were over 330 responses in this survey alone. I’ll be interested to see if other organizations adopt this approach.