Positive results for teaching statistics by computer

Posted on by

A recent study shows that students taking an online statistics course utilizing software from Carnegie Mellon do better than students who take a hybrid course with a classroom classroom:

The study, called “Interactive Learning Online at Public Universities,” involved students taking introductory statistics courses at six (unnamed) public universities. A total of 605 students were randomly assigned to take the course in a “hybrid” format: they met in person with their instructors for one hour a week; otherwise, they worked through lessons and exercises using an artificially intelligent learning platform developed by learning scientists at Carnegie Mellon University’s Open Learning Initiative.

Researchers compared these students against their peers in the traditional-format courses, for which students met with a live instructor for three hours per week, using several measuring sticks: whether they passed the course, their performance on a standardized test (the Comprehensive Assessment of Statistics), and the final exam for the course, which was the same for both sections of the course at each of the universities…

The robotic software did have disadvantages, the researchers found. For one, students found it duller than listening to a live instructor. Some felt as though they had learned less, even if they scored just as well on tests. Engaging students, such as professors might by sprinkling their lectures with personal anecdotes and entertaining asides, remains one area where humans have the upper hand.

But on straight teaching the machines were judged to be as effective, and more efficient, than their personality-having counterparts.

As someone who regularly teaches both Statistics and Social Research (a research methods course), these findings are intriguing. I understand the urge to curb costs while still providing a good education. However, I have three questions that perhaps go beyond these findings:

1. Are there any benefits for students from being in a classroom for three hours a week beyond learning outcomes? Is there a social dimension to the classroom setting that could enhance learning? For example, it is common for professors to have students work in groups or with each other, sometimes with the idea that being able to teach or effectively help another student will increase a student’s learning. Also, I wonder about learning becoming strictly an individualistic activity. Sure, there are ways to do this online (discussion boards, using Skype, etc.) but does this replicate the kind of discussions faculty and students can have in a classroom?

2. Are there any professors in the United States who might secretly welcome not having to teach statistics?

3. Is there a point in a discipline, like statistics, where the difficulty of the subject matter makes it more helpful to have a live instructor? This study looked at introductory stats courses but would the findings be the same if the courses covered more advanced topics that require more “intuition” and “art” than pure steps or facts?

h/t Instapundit

Sharing data among scientists vs. “Big Data”

Posted on by

In a quest to make data available to other researchers to verify research results, researchers have come up against one kind of data that is not made publicly available: “big data” from big Internet firms.

The issue came to a boil last month at a scientific conference in Lyon, France, when three scientists from Google and the University of Cambridge declined to release data they had compiled for a paper on the popularity of YouTube videos in different countries.

The chairman of the conference panel — Bernardo A. Huberman, a physicist who directs the social computing group at HP Labs here — responded angrily. In the future, he said, the conference should not accept papers from authors who did not make their data public. He was greeted by applause from the audience…

At leading social science journals, there are few clear guidelines on data sharing. “The American Journal of Sociology does not at present have a formal position on proprietary data,” its editor, Andrew Abbott, a sociologist at the University of Chicago, wrote in an e-mail. “Nor does it at present have formal policies enforcing the sharing of data.”

The problem is not limited to the social sciences. A recent review found that 44 of 50 leading scientific journals instructed their authors on sharing data but that fewer than 30 percent of the papers they published fully adhered to the instructions. A 2008 review of sharing requirements for genetics data found that 40 of 70 journals surveyed had policies, and that 17 of those were “weak.”

Who will win the battle between proprietary data and science? The article makes it sound like scientists are all on one side, particularly because of an interest in fighting issues like scientific fraud. At the same time, scientific journals don’t seem to be “enforcing” their guidelines or the individual scientists who are publishing in these journals aren’t following these guidelines.

The other side of this debate is not presented in this story: what do these big Internet firms, like Google, Yahoo, and Facebook think about sharing this data? This is not a small issue: these firms are spending a good amount of money on analyzing this data and probably hoping to use it for their own business and research purposes. For example, Microsoft recently set up a lab with several well-known researchers in New York City. Would the social scientists who work in such labs want to insist that the data be open? Should these companies have to open up their proprietary data to satisfy the requirements of the larger scientific community?

I suspect this will be an ongoing issue as social scientists look to analyze more innovative data that big companies have collected and that are more difficult for researchers to collect on their own. Will researchers be willing to forgo sharing this kind of data with the wider scientific community if they can get their hands on unique data?

Economy down, traffic congestion down

Posted on by

A company that tracks traffic congestion suggests that congestion was down in a number of metropolitan areas in 2011 because of the economy:

Of the 100 most populous metro areas, 70 saw declines in traffic congestion while just 30 had increases, says Jim Bak, co-author of the 2011 U.S. Traffic Scorecard for Kirkland, Wash.-based INRIX…

Bak says the data show that the reduction in gridlock on the nation’s roads stems from rising fuel prices, lackluster gains in employment and modest increases in highway capacity because of construction projects completed under the federal stimulus program.

In some cases, the connection between job growth and increased congestion was clear. Cities that outpaced the national average of 1.5% growth in employment experienced some of the biggest increases in traffic congestion: Miami, 2.3% employment growth; Tampa, up 3%; and Houston, up 3.2%.

Cities that had big drops in congestion often were those that saw road construction slow considerably from 2010 to 2011 and those where gasoline prices were well above the national average at the peak in April 2011.

Does this fall into the small category of benefits of the economic crisis of recent years?

I would guess many metropolitan residents would be happy with less congestion but I would also guess they wouldn’t like the tradeoff of fewer jobs and higher gas prices. Of course, there have been discussions for years about how higher gas prices would limit driving. But does higher gas prices necessarily have to align with less growth?

Sociologist is host of “History Detectives”

Posted on by

I ran into an interesting side job for a sociologist: host of History Detectives on PBS. This involves investigating artifacts like an 1864 military discharge letter signed by President Abraham Lincoln:

The first few hours of filming took place in the Grand Army of the Republic Museum, where Versagi talked about how the artifact was found, and then re-enacted the find by pulling a scrap of paper out of a prop box. Taping continued at a park where Versagi would meet “History Detectives” host Tukufu Zuberi, professor and chair of the sociology department at the University of Pennsylvania, to show him the piece of paper. The “reveal” took place in a Springfield resident’s home, where Versagi listened as the PBS host told her the story of the artifact based on their research.

How exactly does a sociologist get this kind of job over historians? Here is how the History Detectives website describes Zuberi’s contributions:

America has a long history of social upheaval and cultural mood swings. These shifts leave clear signs of their passing. The trick is knowing how to read the signs, and interpret their meaning.
Tukufu is an authority on the subject. Under his scrutiny, even subtle signs can yield vital evidence about the events at a mystery’s core.
He also provides the team with a context for their work, relating descriptive accounts of living conditions in that particular place, at that particular time.

Being aware of the social issues, pressures, and problems of the day can sometimes help the team determine the triggers of a past event, and the motives of the people involved.

I also wonder if there isn’t a lot of room for a sociologist to talk about how mysteries develop and are understood by the public. For example, what is the social significance of an Abraham Lincoln artifact and why is Lincoln still so popular today (see an earlier post about another sociologists who tackles this)? Not everything becomes an artifact and there is a lot of work that goes into creating and supporting cultural narratives.

If you want to see a list of episodes Zuberi hosts, they are listed on his CV.

By the way, I am a supporter of having more sociologists positively portrayed on TV and in movies (see earlier posts on this topic here and here).

McMansions are not retirement vehicles

Posted on by

In a discussion of the number of Americans approaching retirement age, one economist briefly mentions one issue: the expectation some retirees had that their homes would be retirement vehicles may be misguided.

“The fact of the matter is that this aging-but-not-yet-aged segment of the baby boomer class can’t afford to retire,” said David A. Rosenberg, the chief economist of Gluskin Sheff, a Canadian firm, noting that overall household net worth was 15 percent lower than at the prerecession peak. “Dreams of the 5,000-square-foot McMansion being a viable retirement asset have morphed into nightmares of a deflationary ball and chain.”

This is indicative of a larger shift: with the ongoing housing crisis, homeowners can’t expect to cash out their homes in the same way they might have in the late 1990s and early 2000s. McMansions are emblematic of people taking out large mortgages and then having difficulty in paying off their mortgages or moving (“downshifting”) to a cheaper residence because of the reduced value of their homes. Of course, this shift moves housing back to its historic role as a dwelling but not necessarily as a golden nest-egg for retirees.

I would be interested in knowing at which age people tend to purchase McMansions or other large homes. For example, buying a McMansion in one’s 50s might not be the best idea if someone wants to retire at 65. If a homeowner is willing to live in a home for a longer period of time, say more than 10-15 years, then such a purchase might not be as unreasonable as the homeowner can pay off more of their mortgage. Living in a home long-term may not be possible for everyone given changing job circumstances as well as the general mobile nature of American society but there are ways to help ensure one could make more money off of selling a large home.

Quick Review: The Great Inversion

Posted on by

I recently read The Great Inversion, a book by Alan Ehrenhalt (see an interview about the book here), about how more Americans are seeking denser living areas. This is not a new idea as plenty of commentators have addressed this in recent years but this book attempts to provide a broad overview of the phenomenon. Here are four thoughts about the urban trends discussed in this book:

1. This book is built around case studies. This is both a strength and weakness. As a strength, Ehrenhalt examines several American cities such as Phoenix, Atlanta, and Denver that don’t get as much attention from urban sociologists. Even as urban sociologists admit that the urban landscape in America has changed a lot since the beginnings of the Chicago School in the early 1900s, most studies examine “traditional” American cities like Chicago, Philadelphia, New York City, and Boston. But these case studies seem more impressionistic than anything else; hard data is difficult to find in this book. There are few figures about how many Americans have actually made the choice to move (versus surveys that suggest Baby Boomers and Millennials desire denser homes). The case studies often look at smaller areas of a metropolitan region, such as the Sheffield neighborhood in Chicago, but don’t address the big picture across regions or throughout the United States.

2. Ehrenhalt is careful to try to straddle the middle line between urbanists and suburbanists (defined a few times as people like Joel Kotkin). But the problem with this is that I don’t think he makes his argument very strongly. Here is what he wants to argue: American urban areas will look quite different in a few decades as more Americans seek out denser housing. However, he doesn’t want to argue this too strongly and backpedals from this at points. Here is his conclusion about Tysons Corner, the last case study of the book:

I’m convinced of that because I see all around me a generation of young, mainstream, middle-class adults who are looking for some form of midlevel urban experience: not bohemian inner-city adventure, but definitely not cul-de-sacs and long automobile commutes. There are more of them coming into the residential market every year. They like the idea of having some space, but they aren’t feeling in terror at the mention of density. They aren’t willing to sell their cars, but they appreciate the advantage of having another way to get around. If Tysons Corner is rebuilt on a reasonable human scale and with a modicum of physical appeal, they will go for it, imperfect as it may be.

And then we will begin to see experiments of this sort in suburbs all over the country, launched by developers and local governments that may still be a little nervous about density but will know one thing for sure: If Tysons Corner can be reborn, nothing in the suburbs is beyond hope. If the effort to rebuild Tysons Corner somehow succeeds, it will become a national model for retrofitting suburbia for the millennial generation.

It is less of an argument that there is a strong push for these options and more of an argument that demographics will change urban forms. This may be correct but it seems like Ehrenhalt seems unwilling to push too hard for this.

3. Ehrenhalt suggests our cities will look more European in a few decades as poorer Americans move to the suburbs and wealthier Americans move back to the cities. This may indeed happen but I think Ehrenhalt generally downplays the cultural factors behind American suburbia and the difficulties that may occur in this demographic inversion (see #4 below).

4. This book reminds me that there are a lot of potentially interesting things that could happen in American suburbs in the coming decades. In particular, the densification of suburbs has the potential to change the character of a number of larger and/or thriving suburbs. Many communities might turn to retrofitting out of desperation in order to start generating tax revenues from vacant properties. However, while Ehrenhalt thinks that demographics will push in this direction, I think there will still be substantial pushback in some places. I’m thinking of a suburb like Naperville, a community that definitely could incorporate high-rises in the downtown and along the I-88 corridor but has thus far resisted big projects. Perhaps circumstances could change but I imagine it might take a while for this to happen.

Debating the reliability of social science research

Posted on by

A philosopher argues social science research is not that reliable and therefore should have a limited impact on public policy:

Without a strong track record of experiments leading to successful predictions, there is seldom a basis for taking social scientific results as definitive.  Jim Manzi, in his recent book, “Uncontrolled,” offers a careful and informed survey of the problems of research in the social sciences and concludes that “nonexperimental social science is not capable of making useful, reliable and nonobvious predictions for the effects of most proposed policy interventions.”

Even if social science were able to greatly increase their use of randomized controlled experiments, Manzi’s judgment is that “it will not be able to adjudicate most policy debates.” Because of the many interrelated causes at work in social systems, many questions are simply “impervious to experimentation.”   But even when we can get reliable experimental results, the causal complexity restricts us to “extremely conditional, statistical statements,” which severely limit the range of cases to which the results apply.

My conclusion is not that our policy discussions should simply ignore social scientific research.  We should, as Manzi himself proposes, find ways of injecting more experimental data into government decisions.  But above all, we need to develop a much better sense of the severely limited reliability of social scientific results.   Media reports of research should pay far more attention to these limitations, and scientists reporting the results need to emphasize what they don’t show as much as what they do.

Given the limited predictive success and the lack of consensus in social sciences, their conclusions can seldom be primary guides to setting policy.  At best, they can supplement the general knowledge, practical experience, good sense and critical intelligence that we can only hope our political leaders will have.

Several quick thoughts:

1. There seems to be some misunderstanding about the differences between the social and natural sciences. The social sciences don’t have laws in the same sense that the natural sciences do. People don’t operate like planets (to pick up on one of the examples). Social behaviors change over time in response to changing conditions and this makes study more difficult.

2. There is a heavy emphasis in this article on experiments. However, these are more difficult to conduct in the social realm: it is hard to control for all sorts of possible influential factors, have a sizable enough N to make generalizations, and experiments in the “harder sciences” like medicine have some of their own issues (see this critique of medical studies).

3. Saying the social sciences have some or a little predictive ability is different than saying they have none. Having some knowledge of social life is better than none when crafting policy, right?

4. Leaders should have “the general knowledge, practical experience, good sense and critical intelligence” to be able to make good decisions. Are these qualities simply individualistic or could social science help inform and create these abilities?

5. While there are limitations to doing social science research, there are also ways that researchers can increase the reliability and validity of studies. These techniques are not inconsequential; there are big differences between good research methods and bad research methods in what kind of data they produce. There is a need within social science to think about “big science” more often rather than pursuing smaller, limited studies but these studies than can speak to broader questions typically require more data and analysis which in turn requires more resources and time.

McMansion owners are bike haters

Posted on by

I know the arguments between drivers and bike riders can become quite heated but I haven’t seen this twist before regarding bike-hating McMansion owners:

It takes just five minutes on top of this bike for me to know I am good for the environment, healthy, frugal, smarter than all of y’all.

Whoa! Slow your roll, Virginia boy. Can’t you see that I’m busy saving the Earth on my bike? That SUV of yours takes up half the city. I bet you live on a huge cul-de-sac, in a McMansion with your own septic system and sad little saplings planted by the developer who chopped down all the mature trees to build that monument to yourself. I bet you don’t even recycle.

I roll my eyes at you, shake my helmet head at your obvious ignorance.

Headline of this column: “Bike lover, bike hater: Depends on whether you’re on four wheels or two.”

I’m always intrigued by the propensity in our culture to label people based on one consumer item, whether it is a McMansion or a bicycle. Here we get a classic description of a McMansion owner: because a person lives in a McMansion, they hate the earth, drive an SUV, and are generally self-centered. Granted, buying a McMansion is a large monetary choice, a home probably the biggest single investment people will make in a lifetime, and large symbolic choice as Americans have long held that one’s home should reflect those who live in them. It would be interesting to see how these single choices, McMansion or bike, line up with other consumer choices: do bike riders live in the city, tend to drive a Prius (or even better, not own a car and utilize Zipcar), and shop at Whole Foods while McMansion owners are suburbanites who tend to drive SUVs and shop at Walmart?

This is a reminder that moral values are often attached to consumer goods. Buying items isn’t simply about functionality or desires but rather indicate how a consumer views the world and what they value. Additionally, certain items, such as McMansions, are clearly viewed as moral signals by others.

 

No, Ikea does not sell a prefabricated home

Posted on by

Two months ago, I noted a story about a prefab home whose interior was outfitted by Ikea. Apparently, lots of people thought Ikea was providing the whole home:

The story went nuts. Literally within an hour, we were getting emails from the Phillippines to South America to Australia to Iceland — it seemed to be real big in Iceland. We heard right away from Greece and Italy — people who were interested in knowing more about an “Ikea house.” We got 2 million Web hits.

It got mentioned on the “Tonight Show” and “Today.” The Huffington Post picked it up but later issued a correction.

We got 4,000 emails in just the first few days after the erroneous story broke. We tried to do our best to respond right away, explaining to people what we do and that the Aktiv house just contains some Ikea products.

Ikea put out a corporate press release explaining things, saying that despite erroneous reports, it is not selling prefabricated homes in the United States. We were told that people were walking into Ikea stores around the world and asking about getting a house.

As I noted two months ago, you can still get a prefabricated home from Menards!

Based on the popularity of this idea, I wonder if Ikea is looking into providing prefabricated houses. I’m sure there would be some issues of economies of scale and getting into home development but I imagine Ikea could tackle these issues if they wanted to.

Another question: are there are other brands that people would buy prefab houses from? What if Pottery Barn got into this?

David Brooks, “Boo-boos in Paradise,” and American public intellectuals

Posted on by

I like David Brooks’ pop sociology analysis of the suburbs in Bobos in Paradise but a piece in Philadelphia suggests Brooks got some of his facts wrong:

Brooks, an agile and engaging writer, was doing what he does best, bringing sweeping social movements to life by zeroing in on what Tom Wolfe called “status detail,” those telling symbols — the Weber Grill, the open-toed sandals with advanced polymer soles — that immediately fix a person in place, time and class. Through his articles, a best-selling book, and now a twice-a-week column in what is arguably journalism’s most prized locale, the New York Times op-ed page, Brooks has become a must-read, charming us into seeing events in the news through his worldview.

There’s just one problem: Many of his generalizations are false. According to Amazon.com sales data, one of Goodwin’s strongest markets has been deep-Red McAllen, Texas. That’s probably not, however, QVC country. “I would guess our audience would skew toward Blue areas of the country,” says Doug Rose, the network’s vice president of merchandising and brand development. “Generally our audience is female suburban baby boomers, and our business skews towards affluent areas.” Rose’s standard PowerPoint presentation of the QVC brand includes a map of one zip code — Beverly Hills, 90210 — covered in little red dots that each represent one QVC customer address, to debunk “the myth that they’re all little old ladies in trailer parks eating bonbons all day.”

But this isn’t the main complaint of this arguement: rather, the main problem is that Brooks is considered a public intellectual and his words have a lot of weight:

On the publication of Bobos, New York Times critic Walter Goodman lumped Brooks with William H. Whyte Jr., author of The Organization Man, and David Riesman, who wrote The Lonely Crowd, as a practitioner of “sociological journalism.” (In the introduction to Bobos, Brooks invoked Whyte — plus Jane Jacobs and John Kenneth Galbraith — as predecessors.) In 2001, the New School for Social Research, in Manhattan, held a panel discussion in which real-life scholars pondered the bobo. When, in 2001, Richard Posner ranked the 100 highest-profile public intellectuals, Brooks came in 85th, just behind Marshall McLuhan at 82nd, and ahead of Garry Wills, Isaiah Berlin and Margaret Mead.

Ironically, Richard Florida is granted the final academic say regarding needing more serious public intellectuals:

Richard Florida, a Carnegie Mellon demographer whose 2002 book The Rise of the Creative Class earned Bobos-like mainstream cachet, nostalgizes an era when readers looked to academia for such insights:

“You had Holly Whyte, who got Jane Jacobs started, Daniel Bell, David Riesman, Galbraith. This is what we’re missing; this is a gap,” Florida says. “Now you have David Brooks as your sociologist, and Al Franken and Michael Moore as your political scientists. Where is the serious public intellectualism of a previous era? It’s the failure of social science to be relevant enough to do it.”

Here is what I take away from this: this writer is worried that Brooks (and other New Journalists) are influencing public opinion and possibly public policy more through impressionistic writing than facts and correctly interpreting data.

This could make for an interesting discussion involving things like the role of columnists and opinion-makers (facts or zeitgeists?), why social scientists and sociologists aren’t seen as public intellectuals, and who should guide public policy anyway. It is interesting to note that the American Sociological Association (ASA) gave David Brooks the Excellence in Reporting of Social Issues Award in 2011. I assume the ASA didn’t just give the award because Brooks discusses sociological research or is of the same political/social persuasion as sociologists.

By the way, having read a lot of David Brooks and Tom Wolfe, I wonder how many commentators would suggest these two are engaging in similar techniques.