Nudging people for good or evil

Posted on April 4, 2017 by legallysociable

Like much in today’s polarized world, perhaps you only like nudging when people are being moved in a direction you agree with:

We are living in an age in which the behavioral sciences have become inescapable. The findings of social psychology and behavioral economics are being employed to determine the news we read, the products we buy, the cultural and intellectual spheres we inhabit, and the human networks, online and in real life, of which we are a part. Aspects of human societies that were formerly guided by habit and tradition, or spontaneity and whim, are now increasingly the intended or unintended consequences of decisions made on the basis of scientific theories of the human mind and human well-being.

The behavioral techniques that are being employed by governments and private corporations do not appeal to our reason; they do not seek to persuade us consciously with information and argument. Rather, these techniques change behavior by appealing to our nonrational motivations, our emotional triggers and unconscious biases. If psychologists could possess a systematic understanding of these nonrational motivations they would have the power to influence the smallest aspects of our lives and the largest aspects of our societies…

But in spite of revealing these deep flaws in our thinking, Lewis supplies a consistently redemptive narrative, insisting that this new psychological knowledge permits us to compensate for human irrationality in ways that can improve human well-being. The field of behavioral economics, a subject pioneered by Richard Thaler and rooted in the work of Kahneman and Tversky, has taken up the task of figuring out how to turn us into better versions of ourselves. If the availability heuristic encourages people to ensure against very unlikely occurrences, “nudges” such as providing vivid reminders of more likely bad outcomes can be used to make their judgments of probability more realistic. If a bias toward the status quo means that people tend not to make changes that would benefit them, for instance by refusing to choose between retirement plans, we can make the more beneficial option available by automatically enrolling people in a plan with the option to withdraw if they choose…

Lewis does not discuss the ways in which the same behavioral science can be used quite deliberately for the purposes of deception and manipulation, though this has been one of its most important applications. Frank Babetski, a CIA Directorate of Intelligence analyst who also holds the Analytical Tradecraft chair at the Sherman Kent School of Intelligence Analysis at the CIA University, has called Kahneman’s Thinking, Fast and Slow a “must read” for intelligence officers.

This seems like a reasonable point – people can be pushed toward positive and negative behavior – but it still leaves a crucial question: who gets to decide what is worth pushing people toward? Is it manipulation when it goes a direction you don’t want but progress when it goes your way? The two major examples of this playing out in society don’t help much; we may wish that big corporations and national politicians have less ability to sway people but this is also part of having a lot of power. (Similarly, power can be used to benefit people or harm them.) Are we more okay with an individual having biases rather than larger social actors (who can coerce a lot more people at the same time)? If so, then it may be harder to have a large society that functions well.

Social psychology can move forward by pursuing more replication

Posted on May 2, 2013 by legallysociable

Here is an argument that a renewed emphasis on replicating studies will help the field of social psychology move beyond some public issues:

Things aren’t quite as bad as they seem, though. Although Nature’s report was headlined “Disputed results a fresh blow for social psychology,” it scarcely noted that there have been some replications of experiments modelled on Dijksterhuis’s phenomenon. His finding could still out turn to be right, if weaker than first thought. More broadly, social priming is just one thread in the very rich fabric of social psychology. The field will survive, even if social priming turns out to have been overrated or an unfortunate detour.

Even if this one particular line of work is under a shroud, it is important not to lose sight of the fact many of the old standbys from social psychology have been endlessly replicated, like the Milgram effect—the old study of obedience in which subjects turned up electrical shocks (or what they thought were electrical shocks) all the way to four hundred and fifty volts, apparently causing great pain to their subjects, simply because they’d been asked to do it. Milgram himself replicated the experiment numerous times, in many different populations, with groups of differing backgrounds. It is still robust (in hands of other researchers) nearly fifty years later. And even today, people are still extending that result; just last week I read about a study in which intrepid experimenters asked whether people might administer electric shocks to robots, under similar circumstances. (Answer: yes.)

More importantly, there is something positive that has come out of the crisis of replicability—something vitally important for all experimental sciences. For years, it was extremely difficult to publish a direct replication, or a failure to replicate an experiment, in a good journal. Throughout my career, and long before it, journals emphasized that new papers have to publish original results; I completely failed to replicate a particular study a few years ago, but at the time didn’t bother to submit it to a journal because I knew few people would be interested. Now, happily, the scientific culture has changed. Since I first mentioned these issues in late December, several leading researchers in psychology have announced major efforts to replicate previous work, and to change the incentives so that scientists can do the right thing without feeling like they are spending time doing something that might not be valued by tenure committees.

The Reproducibility Project, from the Center for Open Science is now underway, with its first white paper on the psychology and sociology of replication itself. Thanks to Daniel Simons and Bobbie Spellman, the journal Perspectives in Psychological Science is now accepting submissions for a new section of each issue devoted to replicability. The journal Social Psychology is planning a special issue on replications for important results in social psychology, and has already received forty proposals. Other journals in neuroscience and medicine are engaged in similar efforts: my N.Y.U. colleague Todd Gureckis just used Amazon’s Mechanical Turk to replicate a wide range of basic results in cognitive psychology. And just last week, Uri Simonsohn released a paper on coping with the famous file-drawer problem, in which failed studies have historically been underreported.

It is a good thing if the social sciences were able to be more sure of their findings. Replication could go a long way to moving the conversation away from headline-grabbing findings based on small Ns to be more certain results that a broader swath of an academic field can agree with. The goal is to get it right in the long run with evidence about human behaviors and attitudes, not necessarily in the short-term.

Even with a renewed emphasis on replication, there might still be some issues:

1. The ability to publish more replication studies would certainly help but is there enough incentive for researchers, particularly those trying to establish themselves, to pursue replication studies over innovative ideas and areas that gain more attention?

2. What about the number of studies that are conducted with WEIRD populations, primarily US undergraduate students? If studies continue to be replicated with skewed populations, is much gained?

Debate over priming effect illustrates need for replication

Posted on February 1, 2013 by legallysociable

A review of the literature regarding the priming effect highlights the need in science for replication:

At the same time, psychology has been beset with scandal and doubt. Formerly high-flying researchers like Diederik Stapel, Marc Hauser, and Dirk Smeesters saw their careers implode after allegations that they had cooked their results and managed to slip them past the supposedly watchful eyes of peer reviewers. Psychology isn’t the only field with fakers, but it has its share. Plus there’s the so-called file-drawer problem, that is, the tendency for researchers to publish their singular successes and ignore their multiple failures, making a fluke look like a breakthrough. Fairly or not, social psychologists are perceived to be less rigorous in their methods, generally not replicating their own or one another’s work, instead pressing on toward the next headline-making outcome.

Much of the criticism has been directed at priming. The definitions get dicey here because the term can refer to a range of phenomena, some of which are grounded in decades of solid evidence—like the “anchoring effect,” which happens, for instance, when a store lists a competitor’s inflated price next to its own to make you think you’re getting a bargain. That works. The studies that raise eyebrows are mostly in an area known as behavioral or goal priming, research that demonstrates how subliminal prompts can make you do all manner of crazy things. A warm mug makes you friendlier. The American flag makes you vote Republican. Fast-food logos make you impatient. A small group of skeptical psychologists—let’s call them the Replicators—have been trying to reproduce some of the most popular priming effects in their own labs.

What have they found? Mostly that they can’t get those results. The studies don’t check out. Something is wrong. And because he is undoubtedly the biggest name in the field, the Replicators have paid special attention to John Bargh and the study that started it all.

While some may find this discouraging, it sounds like the scientific process is being followed. A researcher, Bargh, finds something interesting. Others follow up to see if Bargh was right and to try to extend the idea. Debate ensues once a number of studies have been done. Perhaps there is one stage left to finish off in this process: the research community has to look at the accumulated evidence at some point and decide whether the priming effect exists or not. What does the overall weight of the evidence suggest?

For the replication process to work well, a few things need to happen. Researchers need to be willing to repeat the studies of others as well as their own studies. They need to be willing to report both positive and negative findings, regardless of which side of the debate they are on. Journals need to provide space for positive and negative findings. This incremental process will take time and may not lead to big headlines but its steady approach should pay off in the end.

Social psychologist on quest to find researchers who falsify data

Posted on December 17, 2012 by legallysociable

The latest Atlantic magazine includes a short piece about a social psychologist who is out to catch other researchers who falsify data. Here is part of the story:

Simonsohn initially targeted not flagrant dishonesty, but loose methodology. In a paper called “False-Positive Psychology,” published in the prestigious journal Psychological Science, he and two colleagues—Leif Nelson, a professor at the University of California at Berkeley, and Wharton’s Joseph Simmons—showed that psychologists could all but guarantee an interesting research finding if they were creative enough with their statistics and procedures.

The three social psychologists set up a test experiment, then played by current academic methodologies and widely permissible statistical rules. By going on what amounted to a fishing expedition (that is, by recording many, many variables but reporting only the results that came out to their liking); by failing to establish in advance the number of human subjects in an experiment; and by analyzing the data as they went, so they could end the experiment when the results suited them, they produced a howler of a result, a truly absurd finding. They then ran a series of computer simulations using other experimental data to show that these methods could increase the odds of a false-positive result—a statistical fluke, basically—to nearly two-thirds.

Just as Simonsohn was thinking about how to follow up on the paper, he came across an article that seemed too good to be true. In it, Lawrence Sanna, a professor who’d recently moved from the University of North Carolina to the University of Michigan, claimed to have found that people with a physically high vantage point—a concert stage instead of an orchestra pit—feel and act more “pro-socially.” (He measured sociability partly by, of all things, someone’s willingness to force fellow research subjects to consume painfully spicy hot sauce.) The size of the effect Sanna reported was “out-of-this-world strong, gravity strong—just super-strong,” Simonsohn told me over Chinese food (heavy on the hot sauce) at a restaurant around the corner from his office. As he read the paper, something else struck him, too: the data didn’t seem to vary as widely as you’d expect real-world results to. Imagine a study that calculated male height: if the average man were 5-foot?10, you wouldn’t expect that in every group of male subjects, the average man would always be precisely 5-foot-10. Yet this was exactly the sort of unlikely pattern Simonsohn detected in Sanna’s data…

Simonsohn stressed that there’s a world of difference between data techniques that generate false positives, and fraud, but he said some academic psychologists have, until recently, been dangerously indifferent to both. Outright fraud is probably rare. Data manipulation is undoubtedly more common—and surely extends to other subjects dependent on statistical study, including biomedicine. Worse, sloppy statistics are “like steroids in baseball”: Throughout the affected fields, researchers who are too intellectually honest to use these tricks will publish less, and may perish. Meanwhile, the less fastidious flourish.

The current research may just provide incentives for researchers to cut corners and end up with false results. Publishing is incredibly important for the career of an academic and there is little systematic oversight of a researcher’s data. I’ve written before about ways that data could be made more open but it would take some work to put these ideas into practice.

What I wouldn’t want to happen is have people read a story like this and conclude that fields like social psychology have nothing to offer because who knows how many of the studies might be flawed. I also wonder about the vigilante edge to this story – it makes a journalistic piece to tell about a social psychologist who is battling his own field but this isn’t how science should work. Simonsohn should be joined by others who should also be concerned by these potential issues. Of course, there may not be many incentives to pursue this work as it might invite criticism from inside and outside the discipline.

Don’t dismiss social science research just because of one fradulent scientist

Posted on January 11, 2012 by legallysociable

Andrew Ferguson argued in early December that journalists fall too easily for bad academic research. However, he seems to base much of his argument on the actions of one fraudulent scientist:

Lots of cultural writing these days, in books and magazines and newspapers, relies on the so-called Chump Effect. The Effect is defined by its discoverer, me, as the eagerness of laymen and journalists to swallow whole the claims made by social scientists. Entire journalistic enterprises, whole books from cover to cover, would simply collapse into dust if even a smidgen of skepticism were summoned whenever we read that “scientists say” or “a new study finds” or “research shows” or “data suggest.” Most such claims of social science, we would soon find, fall into one of three categories: the trivial, the dubious, or the flatly untrue.

A rather extreme example of this third option emerged last month when an internationally renowned social psychologist, Diederik Stapel of Tilburg University in the Netherlands, was proved to be a fraud. No jokes, please: This social psychologist is a fraud in the literal, perhaps criminal, and not merely figurative, sense. An investigative committee concluded that Stapel had falsified data in at least “several dozen” of the nearly 150 papers he had published in his extremely prolific career…

But it hardly seems to matter, does it? The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior…

Who cares? The experiments are preposterous. You’d have to be a highly trained social psychologist, or a journalist, to think otherwise. Just for starters, the experiments can never be repeated or their results tested under controlled conditions. The influence of a hundred different variables is impossible to record. The first group of passengers may have little in common with the second group. The groups were too small to yield statistically significant results. The questionnaire is hopelessly imprecise, and so are the measures of racism and homophobia. The notions of “disorder” and “stereotype” are arbitrary—and so on and so on.

Yet the allure of “science” is too strong for our journalists to resist: all those numbers, those equations, those fancy names (say it twice: the Self-Activation Effect), all those experts with Ph.D.’s!

I was afraid that the actions of one scientist might taint the work of many others.

But there are a couple of issues here and several are worth pursuing:

1. The fact that Stapel committed fraud doesn’t mean that all scientists do bad work. Ferguson seems to want to blame other scientists for not knowing Stapel was committing fraud – how exactly would they have known?

2. Ferguson doesn’t seem to like social psychology. He does point to some valid methodological concerns: many studies involve small groups of undergraduates. Drawing large conclusions from these studies is difficult and indeed, perhaps dangerous. But this isn’t all social psychology is about.

2a. More generally, Ferguson could be writing about a lot of disciplines. Medical research tends to start with small groups and then decisions are made. Lots of research, particularly in the social sciences, could be invalidated if Ferguson was completely right. Ferguson really would suggest “Most such claims of social science…fall into one of three categories: the trivial, the dubious, or the flatly untrue.”?

3. I’ve said it before and I’ll say it again: journalists need more training in order to understand what scientific studies mean. Science doesn’t work in the way that journalists suggests where there is a steady stream of big findings. Rather, scientists find something and then others try to replicate the findings in different settings with different populations. Science is more like an accumulation of evidence than a lot of sudden lightning strikes of new facts. One small study of undergraduates may not tell us much but dozens of such experiments among different groups might.

4. I can’t help but wonder if there is a political slant to this: what if scientists were reporting positive things about conservative viewpoints? Ferguson complains that measuring things like racism and homophobia are difficult but this is the nature of studying humans and society. Ferguson just wants to say that it is all “arbitrary” – this is simply throwing up our hands and saying the world is too difficult to comprehend so we might as well quit. If there isn’t a political edge here, perhaps Ferguson is simply anti-science? What science does Ferguson suggest is credible and valid?

In the end, you can’t dismiss all of social psychology because of the actions of one scientist or because journalists are ill-prepared to report on scientific findings.

h/t Instapundit

Why cases of scientific fraud can affect everyone in sociology

Posted on December 3, 2011 by legallysociable

The recent case of a Dutch social psychologist admitting to working with fraudulent data can lead some to paint social psychology or the broader discipline of sociology as problematic:

At the Weekly Standard, Andrew Ferguson looks at the “Chump Effect” that prompts reporters to write up dubious studies uncritically:

The silliness of social psychology doesn’t lie in its questionable research practices but in the research practices that no one thinks to question. The most common working premise of social-psychology research is far-fetched all by itself: The behavior of a statistically insignificant, self-selected number of college students or high schoolers filling out questionnaires and role-playing in a psych lab can reveal scientifically valid truths about human behavior.

And when the research reaches beyond the classroom, it becomes sillier still…

Described in this way, it does seem like there could be real journalistic interest in this study – as a human interest story like the three-legged rooster or the world’s largest rubber band collection. It just doesn’t have any value as a study of abstract truths about human behavior. The telling thing is that the dullest part of Stapel’s work – its ideologically motivated and false claims about sociology – got all the attention, while the spectacle of a lunatic digging up paving stones and giving apples to unlucky commuters at a trash-strewn train station was considered normal.

A good moment for reaction from a conservative perspective: two favorite whipping boys, liberal (and fraudulent!) social scientists plus journalists/the media (uncritical and biased!), can be tackled at once.

Seriously, though: the answer here is not to paint entire academic disciplines as problematic because of one case of fraud. Granted, some of the questions raised are good ones that social scientists themselves have raised recently: how much about human activity can you discover through relatively small sample tests of American undergraduates? But good science is not based on one study anyway. An interesting finding should be corroborated by similar studies done in different places at different times with different people. These multiple tests and observations help establish the reliability and validity of findings. This can be a slow process, another issue in a media landscape where new stories are needed all the time.

This reminds me of Joel Best’s recommendations regarding dealing with statistics. One common option is to simply trust all statistics. Numbers look authoritative, often come from experts, and they can be overwhelming. Just accepting them can be easy. At the other pole is the common option of saying that all statistics are simply interpretation and are manipulated so we can’t trust any of them. No numbers are trustworthy. Neither approaches are good options but they are relatively easy options. The better route to go when dealing with scientific studies is to have the basic skills necessary to understand whether they are good studies or not and how the process of science works. In this case, this would be a great time to call for better training among journalists about scientific studies so they can provide better interpretations for the public.

In the end, when one prominent social psychologist admits to massive fraud, the repercussions might be felt by others in the field for quite a while.

Social psychologists respond to claim of liberal bias in their field

Posted on February 9, 2011 by legallysociable

The New York Times describes a recent speech by a social psychologist arguing that liberals are underrepresented in academia. While this argument is not new to academia (the article cites several studies of recent years saying similar things), it is interesting to note how the social psychologists responded:

The fields of psychology, sociology and anthropology have long attracted liberals, but they became more exclusive after the 1960s, according to Dr. Haidt. “The fight for civil rights and against racism became the sacred cause unifying the left throughout American society, and within the academy,” he said, arguing that this shared morality both “binds and blinds.”

“If a group circles around sacred values, they will evolve into a tribal-moral community,” he said. “They’ll embrace science whenever it supports their sacred values, but they’ll ditch it or distort it as soon as it threatens a sacred value.” It’s easy for social scientists to observe this process in other communities, like the fundamentalist Christians who embrace “intelligent design” while rejecting Darwinism. But academics can be selective, too, as Daniel Patrick Moynihan found in 1965 when he warned about the rise of unmarried parenthood and welfare dependency among blacks — violating the taboo against criticizing victims of racism…

Can social scientists open up to outsiders’ ideas? Dr. Haidt was optimistic enough to title his speech “The Bright Future of Post-Partisan Social Psychology,” urging his colleagues to focus on shared science rather than shared moral values. To overcome taboos, he advised them to subscribe to National Review and to read Thomas Sowell’s “A Conflict of Visions.”

For a tribal-moral community, the social psychologists in Dr. Haidt’s audience seemed refreshingly receptive to his argument. Some said he overstated how liberal the field is, but many agreed it should welcome more ideological diversity. A few even endorsed his call for a new affirmative-action goal: a membership that’s 10 percent conservative by 2020. The society’s executive committee didn’t endorse Dr. Haidt’s numerical goal, but it did vote to put a statement on the group’s home page welcoming psychologists with “diverse perspectives.” It also made a change on the “Diversity Initiatives” page — a two-letter correction of what it called a grammatical glitch, although others might see it as more of a Freudian slip.

In the old version, the society announced that special funds to pay for travel to the annual meeting were available to students belonging to “underrepresented groups (i.e., ethnic or racial minorities, first-generation college students, individuals with a physical disability, and/or lesbian, gay, bisexual, or transgendered students).”

As Dr. Haidt noted in his speech, the “i.e.” implied that this was the exclusive, sacred list of “underrepresented groups.” The society took his suggestion to substitute “e.g.” — a change that leaves it open to other groups, too. Maybe, someday, even to conservatives.

Several questions come to mind:

1. What will social psychologists do about this in the long run? It’s not surprising that the executive committee didn’t support the 10% by 2020 plan but what will they actively do to promote conservative involvement in this discipline?

2. How will the response to this within academia differ from the response outside of academia, particularly among groups who consistently already make noise about academics being too liberal?

3. In the long run, does this liberal bias mean that all or most of research within this field (and others) is not objective or true?

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this: