Monthly Archives: August 2010

The value of using multiple coders

Posted on by

A well-known psychologist from Harvard is in trouble for allegedly reporting false data from laboratory studies. How the allegations surfaced is illustrative of why researchers should have more than just one person looking at data. As reported in the Chronicle of Higher Education, here is what happened after the psychologist and a graduate student coded an experiment involving rhesus monkeys:

According to the document that was provided to The Chronicle, the experiment in question was coded by Mr. Hauser and a research assistant in his laboratory. A second research assistant was asked by Mr. Hauser to analyze the results. When the second research assistant analyzed the first research assistant’s codes, he found that the monkeys didn’t seem to notice the change in pattern. In fact, they looked at the speaker more often when the pattern was the same. In other words, the experiment was a bust.

But Mr. Hauser’s coding showed something else entirely: He found that the monkeys did notice the change in pattern—and, according to his numbers, the results were statistically significant. If his coding was right, the experiment was a big success.

The second research assistant was bothered by the discrepancy. How could two researchers watching the same videotapes arrive at such different conclusions? He suggested to Mr. Hauser that a third researcher should code the results. In an e-mail message to Mr. Hauser, a copy of which was provided to The Chronicle, the research assistant who analyzed the numbers explained his concern. “I don’t feel comfortable analyzing results/publishing data with that kind of skew until we can verify that with a third coder,” he wrote.

A graduate student agreed with the research assistant and joined him in pressing Mr. Hauser to allow the results to be checked, the document given to The Chronicle indicates. But Mr. Hauser resisted, repeatedly arguing against having a third researcher code the videotapes and writing that they should simply go with the data as he had already coded it. After several back-and-forths, it became plain that the professor was annoyed.

These discrepancies in the data led to indications that something similar had happened in other experiments.

Having multiple coders is good for several reasons:

1. Helping to eliminate or catch problems such as these where someone might be tempted to falsify data.

2. To help interpret ambiguous situations.

3. To demonstrate to the broader research community that the results are more than just one person’s conclusions. (This should also be aided by the review process as other researchers look over the work.)

Quick Review: The Canon

Posted on by

When recently at the Field Museum in Chicago, I encountered several books in the bookstore. I tracked down one of them, a former bestseller, down at the library: The Canon: A Whirligig Tour of the Beautiful Basics of Science by Natalie Angier. A few quick thoughts about the book:

1. This book is an overview of the basic building blocks of science (there are the chapters in order): thinking scientifically, probabilities, scale (different sizes), physics, chemistry, evolutionary biology, molecular biology, geology, and astronomy. Angier interviewed a number of scientists and she both quotes and draws upon their ideas. For someone looking for a quick understanding of these subjects, this is a decent find. From this book, one could delve into more specialized writings.

2. Angier is a science writer for the New York Times. While she tries to bring exuberance to the subject, her descriptions and adjectives are often over the top. This floweriness was almost enough to stop me from reading this book at a few points.

3. To me, the most rewarding chapters were the first three. As a social scientist, I could relate to all three of these and plan to bring some of these thoughts to my students. Thinking scientifically is quite different than the normal experience most of us have of building ideas and concepts on anecdotal data.

a. A couple of the ideas stuck out to me. The first is a reminder about scientific theories: while some think a theory means that it isn’t proven yet so it can be disregarded, scientists view theories differently. Theories are explanations that are constantly being built upon and tested but they often represent the best explanations scientists currently have. A theory is not a law.

b. The second was about random data. Angier tells the story of a professor who runs this activity: at the beginning of class, half the students are told to flip a coin 100 times and record the results. The other half of the students are told to make up the results for 100 imaginary coin flips. The professor leaves the room while the students do this. When she returns, she examines the different recordings and most of the time is able to identify which were the real and imaginary results. How? Students don’t quite understand random data; usually after two consecutive heads or tails, they think they have to have the opposite result. In real random data, there can be runs of 6 of 7 heads or tails in a row even as the results tend to average out in the end.

Overall, I liked the content of the book even as I was often irritated with its delivery. For a social scientist, this was a profitable read as it helped me understand subjects far afield.

Predicting working class job growth

Posted on by

Richard Florida (of The Rise of the Creative Class fame) writes at Atlantic.com about where working class jobs will increase in the future.

The largest metro areas are expected to have the greatest amount of blue-collar job growth. Why these places are expected to have this kind of growth is left unexplained.

Overall, Florida describes the situation:

The good news is that the U.S. will continue to create relatively high-paying working class jobs. These jobs will continue to provide good livelihoods for the workers fortunate enough to have them. The bad news is that their rate of growth will be sluggish and not nearly enough to provide the amount of good, family-supporting jobs required to undergird a middle class of lower-skilled workers.

Takeaway: there will be some good blue collar jobs in the future – but they will be limited.

Comparing male and female drivers

Posted on by

A recent study by New York City shed some light on gender differences in driving and traffic behavior:

80 percent of all crashes in a five-year period in which pedestrians were seriously injured or killed involved men who were driving. The imbalance is far too great to be explained away by the predominance of men among bus, livery, taxi and delivery drivers, said Seth Solomonow, a spokesman for the city’s Transportation Department…

The males of the species are not only more dangerous as drivers, they are more likely to be hurt while walking, the city’s study found. More men than women were killed or injured as pedestrians in every age group except among those over 64 (perhaps because women live longer and were overrepresented). Boys 5 to 17 years old ranked first in the absolute number of pedestrian deaths and serious injuries, with 785, more than twice the number of girls in that age range, though elderly people were more vulnerable as a share of the population.

The article suggests that boys and girls learn these behaviors at a young age: boys think it is okay to be more aggressive around the street.

So where exactly do boys pick up this information? From their fathers/role models, the media, watching people drive or walk around? This socialization process would an intriguing one to delve into.

Varying statistics about DNA matches

Posted on by

NewScientist has a story about a criminal case that demonstrates how scientists can disagree about statistics regarding DNA analysis:

The DNA analyst who testified in Smith’s trial said the chances of the DNA coming from someone other than Jackson were 1 in 95,000. But both the prosecution and the analyst’s supervisor said the odds were more like 1 in 47. A later review of the evidence suggested that the chances of the second person’s DNA coming from someone other than Jackson were closer to 1 in 13, while a different statistical method said the chance of seeing this evidence if the DNA came from Jackson is only twice that of the chance of seeing it if it came from someone else…

[W]e show how, even when analysts agree that someone could be a match for a piece of DNA evidence, the statistical weight assigned to that match can vary enormously.

I recall reading something recently that suggested while the public thinks having DNA samples in a criminal case makes the case very clear, this is not necessarily the case. This article suggests is a lot more complicated and it depends on what lab and scientists are looking at the DNA samples.

Dynamic pricing at sporting events

Posted on by

Kevin Arnovitz at Truehoop reports that the New Orleans Hornets are embracing variable pricing for tickets for the upcoming NBA season. But more interesting is the link to a story about tickets sold by the San Francisco Giants, the first team to completely embrace dynamic pricing.

Last season (2009), the Giants played around the concept of dynamic pricing. Based on demand for tickets for each game, the prices in this section of about 2,000 tickets would fluctuate. When I was in San Francisco last August and was looking for Giants tickets, I saw this section online and was intrigued by it. (For the record, I bought tickets in other seats on StubHub which were cheaper than the variably-priced seats.)

Based on the success of this small sample, the Giants went ahead and introduced dynamic pricing for all the tickets in AT&T Park (a beautiful stadium) during the 2010 season. They are the first team to do this and now several other teams are tinkering with the concept on a small scale.

The last occupied high-rise at Cabrini-Green

Posted on by

Mary Schmich of the Chicago Tribune discusses the last occupied high-rise still standing at Cabrini-Green in Chicago: the 1230 N. Burling building.

Soon, there will be no more buildings like this that had come to symbolize poverty after being built in the 1950s and 1960s. The poor and lower-class have been moved out, some to new mixed-income neighborhoods while others have slipped through the cracks of the system. Though these buildings may disappear, the problems once present in them have not.

Proclaiming the end of the “McMansion era”

Posted on by

CNBC reports that the real estate site Trulia.com says “the McMansion era is over.” This is based on evidence that more people want smaller homes:

Just 9 percent of the people surveyed by Trulia said their ideal home size was over 3,200 square feet. Meanwhile, more than one-third said their ideal size was under 2,000 feet.

“That’s something that would’ve been unbelievable just a few years back,” said Pete Flint, CEO and co-founder of Trulia. “Americans are moving away from McMansions.”

The comments echoed those made in June by Kermit Baker, the chief economist at the American Institute of Architects.

“We continue to move away from the McMansion chapter of residential design, with more demand for practicality throughout the home,” Baker said. “There has been a drop off in the popularity of upscale property enhancements such as formal landscaping, decorative water features, tennis courts, and gazebos.”

“McMansions just look and feel out of place today, given the more cautious environment everyone’s living in,” said Paul Bishop, vice president of research for the National Association of Realtors.

And homebuilders are heeding the call: In a survey of builders last year, nine out of 10 said they planned to build smaller or lower-priced homes.

This is interesting information – the McMansion was and is commonly cited as part of the excess of the late 1990s and early 2000s. But I have a few questions and thoughts:

1. We are in the middle of a housing crisis, one that is virtually unprecedented in recent history. Could these results simply be the result of this period? Look at the data over time: Americans since 1950 have progressively wanted larger homes. Might this change as soon as the economy or housing market picks up again?

1a. We would have to wait and see whether this shift might be a longer-term move to an emphasis on quality and appointments rather than sheer space. Since family size has dropped over the years, it makes sense that homes might not get so large. Or perhaps more people subscribe to some green ideas about having a small footprint.

2. There is still some demand for homes over 3,200 square feet. If you look at the Trulia infographics, most people seem to want homes around the 2,000-2,600 square foot range. These are not small homes – they would be slightly smaller than the average size of new homes built in most years of the 2000s and are larger than most American homes built after World War II.

3. This is survey data which gives us some measure of what people want to buy. However, people still have to make choices on the open market – will they turn down larger houses for smaller houses for an extended amount of time?

4. Will home prices go down or stay low in the long run – or will builders make up for having smaller homes with more features that will cost more?

5. There are some questions about whether a downturn in McMansions is part of a larger, more radical shift toward a new kind of suburbia. Perhaps. But even if this were the case, it would take a while for these new developments to be large enough in number to counter the typical views of suburbia and it would also require Americans to develop a new sense of community.

Will the future be ruled by cities or suburbs?

Posted on by

Two commentators disagree in a special issue of Foreign Policy on global cities: one says cities are the places of the future while another says suburbs are key.

1. In Foreign Policy, Parang Khanna discusses global cities, a concept developed by sociologist Saskia Sassen. Khanna suggests such cities are growing to a point where they exceed the ability for nations or the United Nations to control them. The conclusion is that cities are quite important:

What happens in our cities, simply put, matters more than what happens anywhere else. Cities are the world’s experimental laboratories and thus a metaphor for an uncertain age. They are both the cancer and the foundation of our networked world, both virus and antibody. From climate change to poverty and inequality, cities are the problem — and the solution.

2. Joel Kotkin responds and claims a more dispersed population, in suburbs, can lead to better outcomes in areas like generating wealth, less inequality, and a cleaner environment. He suggests this is particular an issue if we encourage large cities in the developing world:

The goal of urban planners should not be to fulfill their own grandiose visions of megacities on a hill, but to meet the needs of the people living in them, particularly those people suffering from overcrowding, environmental misery, and social inequality. When it comes to exporting our notions to the rest of the globe, we must be aware of our own susceptibility to fashionable theories in urban design — because while the West may be able to live with its mistakes, the developing world doesn’t enjoy that luxury.

An interesting debate – both places have their own issues.  One could ask what residents would prefer to live in (both in the developed and developing world): the wealthy and glamorous megacity or the comfortable and affluent suburbs? Or perhaps different nations could have different planning and policy goals? Or perhaps we need some of both cities and suburbs…