Significant vs. substantive differences, urban vs. suburban snow totals edition

Meteorologist Tom Skilling discusses the difference in snowfall between urban and suburban parts of the Chicago region. In doing so, he illustrates the differences between significant and substantive significance:

Dear Tom,
Why do Chicago’s suburbs get more snow in the winter than Chicago itself?
— Matt, Palatine

Dear Matt,
I do not believe that to be the case. For example, the annual snowfall at Midway Airport is 39.3 inches (Midway being closer to the lake than O’Hare); at O’Hare International Airport, it’s 37.6 inches; at Rockford, 38.3 inches. The differences aren’t large, but they are significant nonetheless. Lake Michigan enhancement of snowfall totals and the occurrence of lake-effect snows in locations closer to the lake all argue that more snow will fall with some regularity at lakeside locations.
Please note that these are generalized statements. Individual snow events will not necessarily conform to the “more snow near the lake” phenomenon. However, averaged over a period of many years, lakeside locations receive more snow than inland locations.

Because the weather data is based on decades of data, we can be fairly confident that there is a difference in snowfall between the three locations mentioned. The location nearest the lake, Midway, receives more snow, Rockford, furthest from the lake, receives a little less snow, and O’Hare, in between though much closer to the lake than Rockford, is in the middle.

On the other hand, there is very little substantive difference between these totals. Over the course of an entire year, the spread between the averages of the three locations is only 1.7 inches total. That is not much. It is likely not noticeable to the average resident. Similarly, I can’t imagine municipalities act much differently because of less than two inches of snow spread out over a year.

This illustrates an issue that often arises in doing statistical analysis: a statistical test may show a significant difference or relationship in the population but the actual difference or relationship is hard to notice or not worth acting on. Here, the data shows real differences in snowfall across locations but the real-world effect is limited.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s