Analysis of available properties at Realtor.com leads to this:

Why these places and not others?
“The places that boast the country’s largest average homes tend to be clustered in higher-priced communities either near bigger economic centers or in outdoors destinations,” says George Ratiu, senior economist and manager of economic research for Realtor.com. “On the flip side of the coin, cities [with] average home sizes [that] skew toward the smaller end are located in geographically constrained locations, where natural boundaries meet high-density development.”
But that’s not all that goes into the equation. Many older communities, such as in the Northeast, that were developed for blue-collar workers tend to be filled with smaller homes. Meanwhile, newer developments geared toward white-collar commuters often offer abodes with more square footage and amenities like open kitchens, which were not popular in the 1950s. You’re more likely to find McMansions in these areas.
In other words, larger homes are in wealthier and more sprawling communities.
I imagine zoning might also play a part in this; what kind of single-family homes are possible and/or encouraged? This is tied to historical patterns and policy decisions.
How do land and housing prices factor into this? Most of the communities listed above are not in the most expensive housing markets.