Tag Archives: methodology

Trying to count the social patterns that have not happened yet, AI job takeover edition

Posted on by

It is hard to know how many jobs AI might eliminate when we cannot yet count many jobs eliminated:

Photo by cottonbro studio on Pexels.com

Measurement doesn’t abolish injustice; it rarely even settles arguments. But the act of counting—of trying to see clearly, of committing the government to a shared set of facts—signals an intention to be fair, or at least to be caught trying. Over time, that intention matters. It’s one way a republic earns the right to be believed in.

The BLS remains a small miracle of civilization. It sends out detailed surveys to about 60,000 households and 120,000 businesses and government agencies every month, supplemented by qualitative research it uses to check and occasionally correct its findings. It deserves at least some credit for the scoreboard. America: 250 years without violent class warfare. And you have to appreciate the entertainment value of its minutiae. The BLS is how we know that, in 2024, 44,119 people worked in mobile food services (a.k.a. food trucks), up 907 percent since 2000; that nonveterinary pet care (grooming, training) employed 190,984 people, up 513 percent; and that the United States had almost 100,000 massage therapists, with five times the national concentration in Napa, California.

These and thousands of other BLS statistics describe a society that has grown more prosperous, and a workforce endlessly adaptive to change. But like all statistical bodies, the BLS has its limits. It’s excellent at revealing what has happened and only moderately useful at telling us what’s about to. The data can’t foresee recessions or pandemics—or the arrival of a technology that might do to the workforce what an asteroid did to the dinosaurs…

This was the point Goolsbee wanted to emphasize: Economists are constrained by numbers. And numerically speaking, nothing indicates that AI has had an impact on people’s jobs. “It’s just too early,” he said.

A lack of certainty should not be mistaken for a lack of concern.

This sounds like a classic issue facing those concerned about particular social problems: can the numbers help you build a case that this issue is important and worthy of the attention of others? With all the possible social problems that need attention, having clear data regarding the problem can help make the case to the public and leaders. But, if this is largely speculation regarding AI, how many will act based on that?

Another important factor regarding counting: it is a key way of trying to make sense of a large and complex society. When you have a country with over 330 million residents, 50 states, and numerous important social patterns occurring, having data to look at can help make sense of what is happening on the broad scale. Anecdotes offer little on a large scale; case studies might provide some insight. Having statistics on a society-wide scale is necessary.

A third way to think about this: those who could generate numerical predictions or have small sectors that could provide early data on this could be helpful for others.

The difficulty of measuring the U.S.’s housing needs

Posted on by

How many housing units are needed in the United States? Different sources disagree and they measure the issue in different ways:

Photo by Curtis Adams on Pexels.com

The disparate projections reflect the challenge of quantifying the nation’s housing needs, a puzzle that rests on assumptions about how much a home should cost, how many people it should hold, and how big a footprint it should have…

The U.S. has 146 million homes, Census Bureau data show. Of those, 8.1 million are “doubled up” households, meaning people are sharing space with non-relatives. Zillow’s housing estimate assumes most of those people would prefer having their own place. There also are 3.4 million vacant homes available to rent or buy, the real estate website says. So Zillow economists subtracted the number of available homes from the number of doubled-up households and concluded that the nation needs 4.7 million more homes…

Several analyses zeroed in on two questions: How many homes should be vacant, and how many consumers have delayed striking out on their own because of the cost…

For many economists, that suggests the equation should be: the number of existing households, plus the number of homes that should be vacant, plus the number of households that would naturally come into being if there was enough inventory to lower prices.

This matters for multiple reasons. First, it is helpful to have more accurate estimates. This can help policies intended to help. These are methodological questions; how do we measure what is happening on the ground? Projections that are too high or too low could lead to not addressing the issue or actions that do not have the intended consequences.

Second, the number of units needed matters because it is part of the public discourse about housing. The article describes estimates ranging from 0 units needed to 8-20 million units are needed. When discussing social problems in public discussions, these numbers can influence a sense of urgency. If people hear there are 5 million units needed, are they more likely to act compared to hearing 1 million units are needed?

Third, the numbers are part of a national discussion. Housing needs can vary quite a bit place to place. Housing is often a very local issue. These numbers are about what could be done on a national level which then has affects on local efforts.

These different measurement strategies and results could end up make it harder to reach consensus on what should be done.

Evidence from the past and future that suggests religious revival is not happening in the United States

Posted on by

I appreciate this part of Ryan Burge’s approach to examining whether a religious resurgence is happening in the US: he looks at past patterns and he considers possible changes in the future.

Photo by Arina Krasnikova on Pexels.com

First, the present data:

The General Social Survey, for instance, reported a steady rise in the “nones” between the early 1990s and 2020. In 2018, the figure was 23%, rising to 28% in 2021. The two most recent estimates are slightly lower — 27% in 2022 and 25% in 2024. Similarly, the “headline finding” from Pew’s Religious Landscape Survey was that both the decline of Christianity and the rise of the unaffiliated have paused in recent years.

As Burge and others have noted, recent data seems to show a stopping/slowing/plateauing of two trends: fewer people affiliating with Christian traditions and more Americans identified as “religious nones.”

But these patterns are also related to aging and whose activity researchers can examine. Burge next turns to the future:

When you compare generations, the pattern is obvious. The youngest members of the Silent Generation were born in the early 1940s, and just 7% report no religious affiliation. In less than a decade, they — and a growing share of Baby Boomers (18% unaffiliated) — will disappear from survey samples.

Meanwhile, millennials are moving solidly into middle age, and 36% of them say they have no religion. Generation Z, all of whom will soon be adults, are even less religious: 43% are nones. That’s 25 points higher than the Boomers they’re replacing. So if the overall share of nones sits around 28% now, it will inevitably rise as generational turnover continues.

Could millennials and Gen Z find God in the years ahead? Possibly — but it would require a transformation unlike anything seen in modern times. Roughly 10 million millennials would have to reaffiliate with religion, followed by another 18 million Gen Zers. There’s no sign of that happening in any dataset.

In other words, for the percent of people in the United States to identify as Christian in the future at the same rates as now would require more young people to become Christian. For the percentage of Christians to grow, even more religious change would need to take place.

By looking at past, present, and future possibilities, Burge concludes: “I can say without equivocation that there’s no clear or compelling evidence that younger Americans are more religious than their parents or grandparents.”

When trying to understand what is happening in a social group or society, one data point or set of evidence is often not enough to fully understand what is happening. Patterns can change over time or the way we understand the world can change over time but a compelling case needs to be made. Seizing on new evidence that does not fit what we know about something might hint at significant change – or it could be a sampling outlier. Good steady research can help reveal these patterns even if there are multiple actors wishing that we could have identifiable patterns more quickly.

Analyzing what Americans value by examining lots of obituaries

Posted on by

How are people remembered? One team of researchers analyzed millions of obituaries. Here is the abstract from the recently published study:

Photo by Pavel Danilyuk on Pexels.com

How societies remember the dead can reveal what people value in life. We analyzed 38 million obituaries from the United States to examine how personal values are encoded in individual and collective legacies. Using Schwartz’s theory of basic human values, we found that tradition and benevolence dominated legacy reflections, while values like power and stimulation appeared less frequently. Major cultural events—the terrorist attacks of September 11th, the 2008 financial crisis, and the COVID-19 pandemic—were systematically linked to changes in legacy reflections about personal values, with security declining after 9/11, achievement declining after the financial crisis, and benevolence declining for years after COVID-19 began and, to date, not yet returning to baseline. Gender and age of the deceased were also linked to differences in legacy: Men were remembered more for achievement, power, and conformity, while women were remembered more for benevolence and hedonism. Older people were remembered more for tradition and conformity than younger people. These patterns shifted dynamically across the lifespan, with obituaries for men showing more age-related variation than legacies for women. Our findings reveal how obituaries serve as psychological and cultural time capsules, preserving not just individual legacies, but also indicating what US society values collectively regarding a life well lived.

This sounds like a novel means by which to examine American cultural values. Obituaries are regularly published and are often accessible to many readers. But to collect and analyze millions of obituaries requires particular skills. This is a big data approach.

The study could also raise multiple additional research questions:

  1. How many of these obituaries were written by the deceased or decided upon before death? Does this change the content?
  2. What is the process by which people writing the obituary after death decide on the words to use and values to emphasize?
  3. How much do these values in obituaries match what people say they value in life at different stages before they die?

What society defines as “sinful” and ranking the most sinful cities

Posted on by

A recent Wallethub list of the “most sinful cities in America” is built on this definition of sin:

Photo by Pavel Danilyuk on Pexels.com

“Regardless of any particular religious tenets, certain activities are considered ‘sinful’ by society as a whole. Sometimes, these activities are always bad, like violent crimes or identity theft. In other cases, they may be relatively harmless in moderation but incredibly destructive when not kept under control, such as alcohol use or gambling. The most sinful cities are those where illicit activities and vices alike are the most widespread.” – Chip Lupo, WalletHub Analyst…

To determine the most sinful cities in America, WalletHub compared 182 cities — including the 150 most populated U.S. cities, plus at least two of the most populated cities in each state — across seven key dimensions: 1) Anger & Hatred, 2) Jealousy, 3) Excesses & Vices, 4) Greed, 5) Lust, 6) Vanity and 7) Laziness.

We examined those dimensions using 37 relevant metrics listed below with their corresponding weights. Each metric was graded on a 100-point scale, with a score of 100 representing the highest level of sinfulness.

I find intriguing the idea that sins as defined by American society are less about religious traditions and more about social constructions of sin. Where do these ideas about sin come from and who defines them? The seven categories seem like they could match up with the traditional seven deadly sins.

If Americans see a list about sins, how many connect that to a religious meaning rather than a social meaning? If Americans grow up loosely connected to religion or are not connected at all, how do they learn about sin? Perhaps sin is more like modern capitalism which sociologist Max Weber argued lost it religious motivations and meanings decades ago. Are these measures good proxies for secularized sins?

Looking at the list of cities, some would not be a surprise. Others might be. For example, a number of cities in what would be considered the Bible Belt make the top 10. There are also some cities that some Americans might assume are higher than they are (Washington, D.C., at #35 and San Francisco at #42, for two examples).

The number of people needed to collect important inflation survey data

Posted on by

How many people participate in collecting data for a key inflation survey from the Bureau of Labor Statistics?

Photo by Mark Stebnicki on Pexels.com

The Bureau of Labor Statistics, the office that publishes the inflation rate, told outside economists this week that a hiring freeze at the agency was forcing the survey to cut back on the number of businesses where it checks prices. In last month’s inflation report, which examined prices in April, government statisticians had to use a less precise method for guessing price changes more extensively than they did in the past…

To calculate the inflation rate, hundreds of government workers called enumerators fan out across cities each month to check how much businesses are charging for products like blue jeans and services like accounting, largely by visiting brick-and-mortar stores. Statisticians in Washington, D.C. roll those figures together into the consumer-price index, a data stream that shows how the cost of living is changing for typical Americans.

If the government’s enumerators can’t track down a specific price in a given city, they try to make an educated guess based on a close substitute: say, cargo pants instead of slacks. But in April, with fewer workers on hand to check prices, statisticians had to base their guesses on less comparable products or other regions of the country—a process called “different-cell imputation”—much more often than usual, according to the BLS…

The inflation rate determines how much social-security benefits go up each year, and where federal tax brackets are set. Private-sector contracts such as wage agreements between companies and unions routinely reference the inflation rate. Payments on $2 trillion of inflation-protected federal bonds hinge on the inflation rate, as do yields on standard Treasury bonds. Businesses, investors and policymakers rely on the reading to guide their decisions. The Federal Reserve is laser focused on inflation data when it sets interest rates for the country.

Surveys require a lot of work to put together. Questions and methods need to be thought through and tested. Data needs to be collected. Analysis requires skill. Sharing results and interpretations is important.

The particular issues outlined above seem to have to do with (1) collecting data, which relies on going out and finding prices, and (2) dealing with missing data, which is related to #1 but is an issue for many surveys. If the survey is utilized by a large number of people, the choices made in the survey process can then affect decisions and policies.

It will be interesting to see what happens here. At what point do academics, policymakers, and others decide that the survey data may not be trustworthy? Which government surveys – and there are many – get priority for funding and having enough employees?

A list of the 30 fastest growing wealthy suburbs includes two suburbs with population declines

Posted on by

I recently found a list of wealthy American with the most population growth. But I noticed that the list ends with two suburbs that lost population during the time period of interest (2018-2023). I suspect this might be because how they selected the communities on the list.

Photo by Altaf Shah on Pexels.com

Methodology: For this study, GOBankingRates analyzed suburbs to find the fastest-growing wealthy suburbs in America. First GOBankingRates found the places with a population between 25,000 and 100,000 according to the U.S. Census American Community Survey. The metro area for each location was found and only the metro areas with a population of 1 million or more were kept. With these suburbs isolated, the numerical and percentage change in population from 2018 to 2023 were found for each city using data from the American Community Survey Census from 2018 and 2023. For each location, GOBankingRates found total population, population ages 65 and over, total households, and household median income all sourced from the American Community Survey. Only places with a median household income of $150,000 or more were kept for this study. Using this data the percentage of the population ages 65 and over were calculated. The cost-of-living indexes were sourced from Sperling’s BestPlaces and include the grocery, healthcare, housing, utilities, transportation, and miscellaneous cost of living indexes. Using the cost-of-living indexes and the national average expenditure costs, as sourced from the Bureau of Labor Statistics Consumer Expenditure Survey, the average expenditure cost for each location were calculated. The livability index was sourced from AreaVibes for each location and included as supplemental information. The average single-family home value was sourced from Zillow Home Value Index for November 2024. Using the average single-family home value, assuming a 10% down payment, and using the most recent national average 30-year fixed mortgage rate, as sourced from the Federal Reserve Economic Data, the average mortgage can be calculated. Using the average mortgage and average expenditure costs, the average total monthly and annual cost of living were calculated. The cities were sorted to show the highest percentage population increase first to show the places with the fastest-growing wealthy suburbs in America. All data was collected on and is up to date as of Jan. 6, 2025.

The bigger question is this: how many suburbs in the United States of population 25,000 to 100,000 have median household incomes over $150,000? I suspect this is not a huge list. Hence, there are only 28 suburbs who meet this criteria and grew between 2018 and 2023.

But it may not take much to change the parameters to include more suburban communities on the list. For example:

  1. What if the median household income was $140,000? Is there a strong reason for leaving the cutoff at $150,000?
  2. Why limit the population to communities between 25,000 and 100,000? If the list could includ communities between 10,000 and 100,000, are there now more growing wealthy suburbs?
  3. Limiting the analysis to metropolitan areas with 1 million people reduces the number of possible regions and suburbs. If the cutoff is 1 million people in an MSA, this means a little over 50 regions are included. Lower the region’s population and you would have more suburbs that might meet the criteria.
  4. Change the list from 30 suburbs to 20 and then the last one on the list would have 5% population growth.

Trying to count all the government bodies in Illinois

Posted on by

Different sources provide different counts of government bodies in Illinois:

Photo by Tim Mossholder on Pexels.com

There are so many units of government in Illinois that people can’t even agree on the total because of differences over what technically qualifies as a government body. The U.S. Census Bureau says 6,930, while the Illinois Department of Revenue, which tracks governments authorized to levy property taxes, reports 6,042. The state comptroller’s office lists 8,529, and a study by the Civic Federation tallied the number at 8,923 as of 2019.

Regardless of the exact answer, the number of governments in Illinois outpaces that seen in bigger states, including Texas (which has 5,533, according to the Census Bureau), Pennsylvania (4,851) and California, a state with a population three times the size of Illinois but half as many local government units…

Today the state has more than 5,700 special-purpose governments, including 851 school districts, 861 drainage districts, 838 fire protection districts, 376 library districts, 348 park districts and 320 multi-township tax assessment districts, according to the state comptroller’s office. Many of the state’s nearly 1,400 districts dedicated to roads and bridges have boundaries overlapping its 1,425 townships.

Most of these governments are outside the Chicago region and represent only a sliver of the state’s population. Nearly two-thirds of Illinois residents live in the six-county Chicago metropolitan area. Meanwhile, 51 of Illinois’ 102 counties have fewer than 25,000 residents, and 15 of those have a population under 10,000, according to a 2021 Civic Federation report. About two-thirds of Illinois’ school districts have fewer than 1,000 students enrolled, and there are 26 school districts with fewer than 100 students.

Two figures stand out:

  1. How do the different counters get to numbers so far apart? The difference is roughly 2,000 bodies of government – what exactly is the scope or taxing ability of these bodies? On the national level, who is considered to have an official count in these area?
  2. Americans tend to like local government that responds to local needs. On one hand, all these government bodies are exerting the will of the people to control local activity. On the other hand, this could be viewed as micromanaging. Certainly there are merges that could happen in some of these categories to take advantage of economies of scale and more efficiently serve a slightly larger population? (I have discussed townships before.)

The focus of this long article is the corruption and lack of oversight than can happen because of so many government bodies. The few times a resident might be reminded of all these bodies is when they see a property tax bill or during election season when there are candidates for all sorts of spots in different bodies.

So is one way to interpret the number of government bodies in Illinois is to suggest that the price of corruption is not enough to convince residents and/or local leaders to give up local control?

Counting the hours spent talking about the possibility of merging Chicago area transit agencies

Posted on by

As conversations take place regarding possibly merging transit agencies in Chicagoland, I wondered if it would be possible to count all the hours of talking, making deals, and working out details. What would the number be? I imagine someone working to provide an accurate count or estimate might run into a few methodological issues:

Photo by Ricky Esquivel on Pexels.com

-Which conversations to count? Is this primarily about formal debate in the legislature or public hearings about the possibilities? Can media reports (whether TV, print, radio, others) count as time? Do digital conversations (texting, emails, in particular apps) count?

-How to count less formal conversations. If conversations take place behind the scenes as opposed to in public settings, can they be found or discovered? What kind of work is needed to track these down?

-Are people willing to talk about their talking? Some might be more willing, some less so. Or perhaps people would be more willing to talk after some major decision is made.

-Do we have some ballpark numbers of how many hours go into major decisions in governments or organizations? What is a “typical” range?

Given the scope of possible changes and the implications whether change occurs or not, the process and the time devoted to it could be worthy of study.

Questioning Census population estimates when they show declines in Illinois

Posted on by

A story on Census population estimates for Illinois’ communities includes some pushback against the numbers:

Photo by Thiago Matos on Pexels.com

While the 2020 census counted responses from household surveys, the annual estimates between the 10-year counts are based in part on counting births, deaths, and moves in and out, using the number of tax returns and Medicare filings.

The numbers do not reflect the recent influx of 41,000 migrants bused and flown to Chicago since August 2022. Census methodology does not account for migrant arrivals. Immigrants are typically hard to count because they may be transient, may not speak English and may want to stay under the radar, researchers said.

Oak Lawn Mayor Terry Vorderer, for one, didn’t buy the new estimates, noting that his town has added new townhomes while not losing housing stock…

Gov. J.B. Pritzker’s office also threw water on the results, highlighting past faulty counts made by the Census Bureau.

“For the last decade, the narrative that Illinois is losing population was fed, by what turned out to be, inaccurate annual preliminary estimates,” Pritzker spokesperson Alex Gough said in a statement. “Illinois remains one of the most populous states in the nation and is on the rise.”

International migration — which has risen nationwide — has nearly tripled in Illinois since 2021, Gough said. The state is in the process of challenging census data to ensure it receives adequate federal funding for programs like Medicare, affordable housing and homeland security, he added.

Is this about methods for counting populations or is this more about politics? For better or worse, these annual estimates have become media stories. Some places are gaining residents, others are losing. Communities with population loss have a hard time shaking all the associations that come with it. The implication is that population loss indicates decline and problems while growth is good.

On the other side, measuring populations is a sizable task. This is why so much effort is expended every ten years. The annual estimates have their own methodologies. They are estimates. This means there is some margin of error. These margins of error should be reported, even if the emphasis in the media continues to be on a concrete number of people gained or lost.

Census numbers might not be perfect but I would be interested in seeing the compelling evidence to suggest their estimates of population declines in some Illinois communities are far off or completely wrong.