Recent quote on doing agendaless science

Vaclav Smil’s How The World Really Works offers an analysis of foundational materials and processes behind life in 2022 and what these portend for the near future. It also includes this as the second to last paragraph of the book:

Is it possible to have no agenda in carrying out analysis and writing such an overview?

Much of what Smil describes and then extrapolates from could be viewed as having an agenda. This agenda could be scientism. On one hand, he reveals some of the key forces at work in our world and on the other hand he provides interpretation of what these mean now and at other times. The writing suggests he knows this; he makes similar points to that quoted above throughout the book to address the issue.

I feel this tension when teaching Research Methods in Sociology. Sociology has a stream of positivism and scientific analysis from its early days, wanting to apply a more dispassionate method to the social world. It has also has early strands of a different approach less beholden to the scientific method and allowing for additional forms of social analysis. These strands continue today and make for an interesting challenge and opportunity in teaching a plurality of methods within a single discipline.

I learned multiple things from the book. I also will need time to ponder the implications and the approach.

Science joke that leads to sociology

A review of a sci-fi novel uses a joke to help make the argument that sociological issues underlying scientific and technological issues:

There’s an old joke in science: Applied physics is called chemistry, applied chemistry is called biology, and applied biology is called sociology.

The purpose of the joke?

Each is more complex, vague, and unpredictable than the last. The problems Robinson lays out proceed neatly along that ladder of intricacy.

Scientific knowledge and technological breakthroughs may not matter much if social groups don’t know what to do with them or use them well to help benefit society.

Getting the data to model society like we model the natural world

A recent session at the American Association for the Advancement of Science included a discussion of how to model the social world:

Dirk Helbing was speaking at a session entitled “Predictability: from physical to data sciences”. This was an opportunity for participating scientists to share ways in which they have applied statistical methodologies they usually use in the physical sciences to issues which are more ‘societal’ in nature. Examples stretched from use of Twitter data to accurately predict where a person is at any moment of each day, to use of social network data in identifying the tipping point at which opinions held by a minority of committed individuals influence the majority view (essentially looking at how new social movements develop) through to reducing travel time across an entire road system by analysing mobile phone and GIS (Geographical Information Systems) data…

With their eye on the big picture, Dr Helbing and multidisciplinary colleagues are collaborating on FuturICT, a 10-year, 1 billion EUR programme which, starting in 2013, is set to explore social and economic life on earth to create a huge computer simulation intended to simulate the interactions of all aspects of social and physical processes on the planet. This open resource will be available to us all and particularly targeted at policy and decision makers. The simulation will make clear the conditions and mechanisms underpinning systemic instabilities in areas as diverse as finance, security, health, the environment and crime. It is hoped that knowing why and being able to see how global crises and social breakdown happen, will mean that we will be able to prevent or mitigate them.

Modelling so many complex matters will take time but in the future, we should be able to use tools to predict collective social phenomena as confidently as we predict physical pheno[men]a such as the weather now.

This will require a tremendous amount of data. It may also require asking for a lot more data from individual members of society in a way that has not happened yet. To this point, individuals have been willing to volunteer information in places like Facebook and Twitter but we will need much more consistent information than that to truly develop models like are suggested here. Additionally, once that minute to minute information is collected, it needs to be put in a central dataset or location to see all the possible connections. Who is going to keep and police this information? People might be convinced to participate if they could see the payoff. A social model will be able to do what exactly – limit or stop crime or wars? Help reduce discrimination? Thus, getting the data from people might be as much of a problem as knowing what to do with it once it is obtained.

Science more about consensus than proven facts

A new book titled The Half-Life of Facts looks at how science is more about consensus than canon. A book review in the Wall Street Journal summarizes the argument:

Knowledge, then, is less a canon than a consensus in a state of constant disruption. Part of the disruption has to do with error and its correction, but another part with simple newness—outright discoveries or new modes of classification and analysis, often enabled by technology. A single chapter in “The Half-Life of Facts” looking at the velocity of knowledge growth starts with the author’s first long computer download—a document containing Plato’s “Republic”—journeys through the rapid rise of the “@” symbol, introduces Moore’s Law describing the growth rate of computing power, and discusses the relevance of Clayton Christensen’s theory of disruptive innovation. Mr. Arbesman illustrates the speed of technological advancement with examples ranging from the magnetic properties of iron—it has become twice as magnetic every five years as purification techniques have improved—to the average distance of daily travel in France, which has exponentially increased over the past two centuries.

To cover so much ground in a scant 200 pages, Mr. Arbesman inevitably sacrifices detail and resolution. And to persuade us that facts change in mathematically predictable ways, he seems to overstate the predictive power of mathematical extrapolation. Still, he does show us convincingly that knowledge changes and that scientific facts are rarely as solid as they appear…

More commonly, however, changes in scientific facts reflect the way that science is done. Mr. Arbesman describes the “Decline Effect”—the tendency of an original scientific publication to present results that seem far more compelling than those of later studies. Such a tendency has been documented in the medical literature over the past decade by John Ioannidis, a researcher at Stanford, in areas as diverse as HIV therapy, angioplasty and stroke treatment. The cause of the decline may well be a potent combination of random chance (generating an excessively impressive result) and publication bias (leading positive results to get preferentially published)…

Science, Mr. Arbesman observes, is a “terribly human endeavor.” Knowledge grows but carries with it uncertainty and error; today’s scientific doctrine may become tomorrow’s cautionary tale. What is to be done? The right response, according to Mr. Arbesman, is to embrace change rather than fight it. “Far better than learning facts is learning how to adapt to changing facts,” he says. “Stop memorizing things . . . memories can be outsourced to the cloud.” In other words: In a world of information flux, it isn’t what you know that counts—it is how efficiently you can refresh.

To add to the conclusion of this review as cited above, it is less about the specific content of the scientific facts and more about the scientific method one uses to arrive at scientific conclusions. There is a reason the scientific process is taught starting in grade school: the process is supposed to help observers get around their own biases and truly observe reality in a reliable and valid way. Of course, whether our bias can actually be eliminated and how we go about observing both matter for our results but it is the process itself that remains intact.

This also gets to an issue some colleagues and I have noticed where college students talk about “proving” things about the world (natural or social). The language of “proof” implies that data collection and analysis can yield unchanging facts which cannot be disputed. But, as this book points out, this is not how science works. When a researcher finds something interesting, they report on their finding and then others go about retesting the findings or applying the findings to new areas. Over time, knowledge accumulates. To put it in the terms of this review, a consensus is eventually reached. But, new information can counteract this consensus and the paradigm building process starts over again (a la Thomas Kuhn in The Structure of Scientific Revolutions). This doesn’t mean science can’t tell us anything but it does mean that the theories and findings of science can change over time (and here is another interesting discussion point: what exactly is a law, theory, and a finding).

In the end, science requires a longer view. As I’ve noted before, the media tends to play up new scientific findings but we are better served looking at the big picture of scientific findings and waiting for a consensus to emerge.

Debating the reliability of social science research

A philosopher argues social science research is not that reliable and therefore should have a limited impact on public policy:

Without a strong track record of experiments leading to successful predictions, there is seldom a basis for taking social scientific results as definitive.  Jim Manzi, in his recent book, “Uncontrolled,” offers a careful and informed survey of the problems of research in the social sciences and concludes that “nonexperimental social science is not capable of making useful, reliable and nonobvious predictions for the effects of most proposed policy interventions.”

Even if social science were able to greatly increase their use of randomized controlled experiments, Manzi’s judgment is that “it will not be able to adjudicate most policy debates.” Because of the many interrelated causes at work in social systems, many questions are simply “impervious to experimentation.”   But even when we can get reliable experimental results, the causal complexity restricts us to “extremely conditional, statistical statements,” which severely limit the range of cases to which the results apply.

My conclusion is not that our policy discussions should simply ignore social scientific research.  We should, as Manzi himself proposes, find ways of injecting more experimental data into government decisions.  But above all, we need to develop a much better sense of the severely limited reliability of social scientific results.   Media reports of research should pay far more attention to these limitations, and scientists reporting the results need to emphasize what they don’t show as much as what they do.

Given the limited predictive success and the lack of consensus in social sciences, their conclusions can seldom be primary guides to setting policy.  At best, they can supplement the general knowledge, practical experience, good sense and critical intelligence that we can only hope our political leaders will have.

Several quick thoughts:

1. There seems to be some misunderstanding about the differences between the social and natural sciences. The social sciences don’t have laws in the same sense that the natural sciences do. People don’t operate like planets (to pick up on one of the examples). Social behaviors change over time in response to changing conditions and this makes study more difficult.

2. There is a heavy emphasis in this article on experiments. However, these are more difficult to conduct in the social realm: it is hard to control for all sorts of possible influential factors, have a sizable enough N to make generalizations, and experiments in the “harder sciences” like medicine have some of their own issues (see this critique of medical studies).

3. Saying the social sciences have some or a little predictive ability is different than saying they have none. Having some knowledge of social life is better than none when crafting policy, right?

4. Leaders should have “the general knowledge, practical experience, good sense and critical intelligence” to be able to make good decisions. Are these qualities simply individualistic or could social science help inform and create these abilities?

5. While there are limitations to doing social science research, there are also ways that researchers can increase the reliability and validity of studies. These techniques are not inconsequential; there are big differences between good research methods and bad research methods in what kind of data they produce. There is a need within social science to think about “big science” more often rather than pursuing smaller, limited studies but these studies than can speak to broader questions typically require more data and analysis which in turn requires more resources and time.

Sociologist discusses the Living Earth Simulator

A sociologist explains a little bit more about the Living Earth Simulator that aims to model society:

Time travel: probably not going to happen any time soon. At least, not in the physical, “Back to the Future” sense. But that doesn’t stop us from trying to peek into the future. In the December issue of Scientific American, writer David Weinberger chats with Dirk Helbing, a Swiss physicist and sociologist who is pitching a project called the Living Earth Simulator, a billion-euro computer system that would absorb vast amounts of data, use it to model global-scale systems — economies, governments, etc. — and predict the future.

Well, maybe. Weinberger speaks with researchers who point out the roadblocks. While it’s possible to model small systems, such as highway and pedestrian traffic, getting a read on the economy, the environment and public health all at once is a much more complicated process. For instance, how would you account for feedback loops in the system — that is, what happens when the computer model’s conclusions alter the situation that it’s modeling? And if you can’t understand the process through which the model generates an answer, the whole thing is just a giant Magic 8 Ball, anyway. The computer may call upon world leaders to “set fire to all the world’s oil wells,” writes Weinberger. “That will not be actionable advice if the policymaker cannot explain why it’s right.”

So data mining will not be encouraged or will the model’s supervisors insist that every discovered pattern come with an explanation?

Interestingly, Helbing is also featured in a recent article in The Economist about pedestrian traffic:

In 1995 Mr Helbing and Peter Molnar, both physicists, came up with a “social force” computer model that used insights from the way that particles in fluids and gases behave to describe pedestrian movement. The model assumed that people are attracted by some things, such as the destination they are heading for, and repelled by others, such as another pedestrian in their path. It proved its worth by predicting several self-organising effects among crowds that are visible in real life.

One is the propensity of dense crowds spontaneously to break into lanes that allow people to move more efficiently in opposing directions. Individuals do not have to negotiate their way through a series of encounters with oncoming people; they can just follow the person in front. That works better than trying to overtake. Research by Mr Moussaid suggests that the effect of one person trying to walk faster than the people around them in a dense crowd is to force an opposing lane of pedestrians to split in two, which has the effect of breaking up the lane next door, and so on. Everyone moves slower as a result.

Two quick thoughts:

1. Combining physics and sociology to explain social behavior seems to be growing in popularity. Here is what I assume: the physics side brings experience in dealing with complex models and a more naturalistic way of explaining human behavior while sociologists bring more theories and knowledge about human contingencies. (But I could be wrong.) It does seem like the combination of these two disciplines could uniquely bridge the gap between the natural and social sciences.

2. Overall, I assume there will be many more projects like this. Getting the data is not so much a problem and we have the computing power to calculate complex models. If this does increase, this will mean some changes within the discipline of sociology: a shift toward mathematical sociology (making regression look relatively simple), thinking about “natural laws” in a way that sociology has generally avoided, and viewing the world in a different way (individuals operating within complex systems).

Space for sociological factors when looking at scientific research

I ran into this blog post discussing a recent study published in Hormones and Behavior titled “Maternal tendencies in women are associated with estrogen levels and facial femininity.” This particular blogger at Scientific American starts out by suggesting she doesn’t like the results:

Friend of the blog Cackle of Rad was the first person to send me this paper, and when I first tried to read it, I got…pretty angry. Being a rather obsessively logical person, I know why I felt angry about this paper, and I worked very hard to step back from it and approach it in a thoroughly scientific manner.

It didn’t work, I called in Kate. That helped a little.

In the end, it’s not a bad paper. The data are the data, as my graduate advisor always says. But data need to be interpreted, and interpretations require context. And I think what’s missing from this paper is not data or adequate methods. It’s context.

In the end, the blogger suggests the “context” needed really are a number of sociological factors that might influence perceptions:

So I wonder if the authors should make more effort to look into sociological factors. How does the intense pressure on women to become wives and mothers change as a function of how feminine the girl looks? I think you can’t separate any of this from this whole “women with higher estrogen want to be mothers” idea. This is why papers like this bug me, because they try to sell this as a evolutionary thing, without really acknowledging how much sociological pressure goes in to making women want to be mothers. And of course now I read them and I instantly get bristly, because what I see is people making assumptions about what I want, and what I must feel like, based on a few aspects of my physiology. It can be of value scientifically…but I don’t want it to apply to ME. I know it might be science, but I also find it more than a bit insulting.

I don’t know this area of research so I don’t have much room to dispute the results of the original study. However, how this blogger goes about this argument for adding sociological factors is interesting. Here are two possible options for making this argument:

1. Argument #1: the study actually could benefit from sociological factors. Definitions of femininity are wrapped up in cultural assumptions and patterns. There is a lot of research to back this up and perhaps we can point to specific parts of this study that would be altered if context was taken into account. But this doesn’t seem to be conclusion of this blog post.

2. Argument #2: there must be some sociological factors involved here because I don’t like these results. On one hand, perhaps it is admirable to admit one doesn’t like these research results. This can often be true about scientific results: it challenges our personal understandings of the world. So why end the post by again emphasizing that the blogger doesn’t like the results? Does this simply reduce sociology to the backup science that one only calls in to suggest that everything is cultural or relative or socially conditioned?

Perhaps I am simply reading too much into this. I don’t know how much natural science research could be improved by including sociological factors, whether it is often considered, or whether this is simply an unusual blog post. Argument #1 is the stronger scientific argument and is the one that should be emphasized more here.

The difference between a sociologist and a geologist, the “soft” and “hard” sciences

Comments about sociology can come from anywhere. See this example from a House member discussing FDA guidelines:

The most intense reaction was generated by a provision offered by Rep. Denny Rehberg (R-Mont.) that would block the FDA from issuing rules or guidance unless its decisions are based on “hard science” rather than “cost and consumer behavior.” The amendment would prevent the FDA from restricting a substance unless it caused greater harm to health than a product not containing the substance.

“The FDA is starting to use soft sciences in some considerations in the promulgation of its rules,” said Rehberg, who defined “hard science”, as “perceived as being more scientific, rigorous and accurate” than behavioral and social sciences.

“I hate to try and define the difference between a psychiatrist and a psychologist, between a sociologist and a geologist, but there is clearly a difference,” he told the committee.

Three sets of comparisons are made here: between psychology and psychiatry, sociology and geology, and “hard” and “soft” science. I think it is pretty easy to make the first two distinctions, particularly between geology and sociology. But the third comparison seems a little strange: does Rehberg want to suggest that soft sciences are less true or that they matter less/are less valid for FDA decision making?

Overall, it sounds like Rehberg is suggesting that the “soft” sciences (psychology and sociology) are not as important in crafting FDA policies as the actual science that says whether certain products are good or bad for humans. But it seems somewhat silly to suggest that perceptions and behaviors shouldn’t influence policy decisions. A lot of legislation is driven by perceptions and values in addition to the actual influences in the physical world. Think about some of the major issues being discussed today such as the deficit or taxes: less of the conversation is about the actual impact on the country and more involves ideologies about who should be responsible for funding the government and what is the proper role and/or size of the government. One of the problems presented in this article is instructive: cigarettes are not illegal and yet government bodies are interested in limiting the consumption of them. Therefore, while menthol cigarettes may not be that much more harmful, if it is attractive to younger kids who then take smoking, why not regulate this? Of course, the smoking example is a loaded one and it would be hard to find someone who would suggest more smoking among teenagers is a good thing.

Based on this discussion, would either political party be willing to create legislation only based on “hard science” or is this only a suggestion when the “hard science” supports one’s existing viewpoint? Additionally, are there politicians out there who have publicly supported sociology rather than suggested it is a “soft” science?

Quick Review: The Canon

When recently at the Field Museum in Chicago, I encountered several books in the bookstore. I tracked down one of them, a former bestseller, down at the library: The Canon: A Whirligig Tour of the Beautiful Basics of Science by Natalie Angier. A few quick thoughts about the book:

1. This book is an overview of the basic building blocks of science (there are the chapters in order): thinking scientifically, probabilities, scale (different sizes), physics, chemistry, evolutionary biology, molecular biology, geology, and astronomy. Angier interviewed a number of scientists and she both quotes and draws upon their ideas. For someone looking for a quick understanding of these subjects, this is a decent find. From this book, one could delve into more specialized writings.

2. Angier is a science writer for the New York Times. While she tries to bring exuberance to the subject, her descriptions and adjectives are often over the top. This floweriness was almost enough to stop me from reading this book at a few points.

3. To me, the most rewarding chapters were the first three. As a social scientist, I could relate to all three of these and plan to bring some of these thoughts to my students. Thinking scientifically is quite different than the normal experience most of us have of building ideas and concepts on anecdotal data.

a. A couple of the ideas stuck out to me. The first is a reminder about scientific theories: while some think a theory means that it isn’t proven yet so it can be disregarded, scientists view theories differently. Theories are explanations that are constantly being built upon and tested but they often represent the best explanations scientists currently have. A theory is not a law.

b. The second was about random data. Angier tells the story of a professor who runs this activity: at the beginning of class, half the students are told to flip a coin 100 times and record the results. The other half of the students are told to make up the results for 100 imaginary coin flips. The professor leaves the room while the students do this. When she returns, she examines the different recordings and most of the time is able to identify which were the real and imaginary results. How? Students don’t quite understand random data; usually after two consecutive heads or tails, they think they have to have the opposite result. In real random data, there can be runs of 6 of 7 heads or tails in a row even as the results tend to average out in the end.

Overall, I liked the content of the book even as I was often irritated with its delivery. For a social scientist, this was a profitable read as it helped me understand subjects far afield.