I noted earlier this week that Yahoo and Facebook are conducting an experiment to see how interconnected people are the world are. Here are some more information about the experiment that was revealed in an interview with Yahoo sociologist/research scientist Duncan Watts:
- On the quality of Facebook’s data:
Cameron Marlow, Facebook’s research scientist and “in-house sociologist,” said that because Facebook’s social graph is essentially the best representation of real world relationships available, “our data can speak more definitively to this question than anything else in history.
Facebook has a treasure trove of information that could be the source of some fascinating research. Does this study signal the start of a new era where researchers will be able to have access to profiles? Will Facebook users, often worried about privacy, stand for this?
- Watts on the problems in past replications of Milgram’s original experiment:
The problem that all of the experiments have had—and the problem that we’re trying to address with this one—is that you never really know what the ground truth is. You know that there’s some network out there involved that connects people, and you know that messages are being passed along on top of this network. The problem is because you can’t see the network underneath them, you don’t know whether people are making the right choices, you don’t know if the chains are as short as possible, and you don’t know why the chains that aren’t completing are stopping.
The major difference here is that Facebook [is] the network over which these messages are being passed. We can see through Facebook how everyone is really connected to everyone else. We can see whether people can actually find these short paths. In previous experiments you were missing this background picture, but now we have the background and we can run the experiment on top of it.
It sounds like past experiments allowed researchers to see the outcome – how many letters reached the target – but didn’t allow them to trace out the paths, either successful or unsuccessful. Being able to see behind the curtain could also reveal some insights about the speeds of certain networks.
- On whether the data is representative:
There are two issues here. You might be concerned that the Facebook network is somehow an unrepresentative sample of the real social graph of the world. The other concern is the people participating in the sample might be an unrepresentative sample of Facebook. I’m not worried about the first concern. Facebook has 750 million users. If it works on Facebook, it’s increasingly difficult to argue that it wouldn’t work for the rest of the world. But the second problem is one that we’re concerned about. It’s really just a matter of getting a broad enough recruiting effort.
I bet there are people who could make a good case that this data is not representative. These same issues plague web surveys: who has consistent access to the Internet and who can be recruited? I would guess that Facebook still skews younger and more educated than the general population.
Watts suggests the results will be published in an academic journal. It will be interesting to read about the outcome and how this is viewed by academics.