Discovering fake randomness

In the midst of a story involving fake data generated for DailyKos by the polling firm, Research 2000, TechDirt summarizes how exactly it was discovered that Research 2000 was faking the data. Several statisticians approached Kos after seeing some irregularities in cross-tab (table) data. The summary and the original analysis on DailyKos are fascinating: even truly random data follows certain parameters. One takeaway: faking random data is a lot harder than it looks. Another takeaway (for me at least): statistics can be both useful and enjoyable.

The three issues as summarized on DailyKos:

Issue one: astronomically low odds that both male and female figures would both be even or odd numbers.

In one respect, however, the numbers for M and F do not differ: if one is even, so is the other, and likewise for odd. Given that the M and F results usually differ, knowing that say 43% of M were favorable (Fav) to Obama gives essentially no clue as to whether say 59% or say 60% of F would be. Thus knowing whether M Fav is even or odd tells us essentially nothing about whether F Fav would be even or odd.

Issue two: the margin between favorability and unfavorability ratings did not display enough variance. If the polls were truly working with random samples, there would be broader range of values.

What little variation there was in the difference of those cross-tab margins seemed to happen slowly over many weeks, not like the week-to-week random jitter expected for real statistics.

Issue three: the changes in favorability ratings from week to week were too random. In most polls like this that track week to week, the most common result is no change. Research 2000 results had too many changes from week to week – often small changes, a percent either way.

For each individual issue, the odds are quite low that each would arise with truly random data. Put all three together happening with the same data and the odds are even lower.

Besides issues regarding integrity of data collection (and it becomes clearer why many people harbor a distrust toward polls and statistics), this is a great example of statistical detective work. Too often, many of us see numbers and quickly trust them (or distrust them). In reality, it takes just a little work to dig deeper into figures to discover what exactly is being measured and how it is being measured. The “what” and “how” matter tremendously as they can radically alter the interpretation of the data. Citizens and journalists need some of these abilities to decipher all the numbers we encounter on a daily basis.

Why vote against honoring sports teams?

Amidst the story of the US House voting 395-5 to honor the Stanley Cup Champion Chicago Blackhawks with a resolution, three of the five who voted “no” explained their vote to the Chicago Tribune.

One was a diehard Flyers fan and Philadelphia native. A second is from New Jersey, across the river from Philly, and said his vote would not line up with his constituent’s interests.

The third “no” vote came from Marion Berry in Arkansas. His explanation:

I am generally opposed to congressional resolutions congratulating sports teams when they are the only reason members have been required to return to Washington to vote for that day. While the success in any sporting event is a source of great pride for all who played a role in the victory and their supporters, these resolutions are far less urgent than the many other important challenges facing our nation, such as job creation and our economy.

While Berry is certainly correct about the relative importance of this resolution, does it matter if he is the only House member who feels this way? Will any of his constituents take note? Is it the sort of fact that can be used for him on the campaign trail – or will he be seen as a killjoy? A quick perusal of the early comments on the story suggest Berry may be on to something…

Blood gadgets

Many consumers don’t ask, and presumably often don’t care, how their newly purchased products came to be. Certain products have drawn attention, such as “blood diamonds” (accompanied by a preachy Hollywood film by the same name) or Nike shoes made by sweatshop laborers.

New York Times columnist Nicholas Kristof draws attention to another set of consumer goods: high-tech gadgets including cell phones. These devices often include hard to obtain minerals, such as tantalum which is found in Congo. There are some activists who are planning to bring attention to this by taking their argument to tech companies like Microsoft, Apple, and Intel:

A humorous new video taunting Apple and PC computers alike goes online this weekend on YouTube, with hopes that it will go viral. Put together by a group of Hollywood actors, it’s a spoof on the famous “I’m a Mac”/”I’m a PC” ad and suggests that both are sometimes built from conflict minerals.

“Guess we have some things in common after all,” Mac admits.

Stay tuned. A strong-enough consumer/activist push will likely lead to these companies pledging to use responsible materials.

Also: one wonders how this decades-long situation in mineral-rich Congo might inform decision-making regarding recent finds of vast amounts of valuable minerals in Afghanistan.

Facebook makes divorce cases easier

Facebook doesn’t just connect friends – it also apparently makes divorce cases easier for many lawyers. According to the Academy of Matrimonial Lawyers:

81 percent of its members have used or faced evidence plucked from Facebook, MySpace, Twitter and other social networking sites, including YouTube and LinkedIn, over the last five years.

The article contains some interesting examples of participants saying one thing in court or to lawyers and then displaying something completely different in the online realm. It is a reminder that the online world is hardly private.

Stealing a little at a time can add up to a lot

Wired reports on a micro charge scheme recently discovered by the Federal Trade Commission. Scammers made charges from false companies on over a million consumer debit and credit cards  in the range of $.20 to $10. These small charges, usually only once to each card, escaped detection for several years but netted almost $10 million for the scammers.

The article also includes a list of the companies used by the scammers for the charges.

54 years ago: Federal interstates are born

On June 29, 1956, President Eisenhower signed the Federal Aid Highway Act of 1956. This legislation, though immediately about infrastructure, had a tremendous impact on American life. Many of the interstate highways of today were built with this money.

These roads have produced a number of changes:

-Suburbanization. People could now easily travel from suburbs to the city center. By the 1960s, many businesses were also locating headquarters along suburban highway exits.

-The American love of the car. This already existed before Federal Interstates but it was enhanced by these well-maintained roads. Now, the average American could drive farther and more safely. From this point on, money for public transportation would always be limited compared to funds for roads.

-Shipping. Many goods today are carried by trucks. Cheap roads coupled with cheap gasoline helps keep Wal-Marts and McDonald’s stocked and cheap.

-Urban renewal. A number of big city neighborhoods were bulldozed to make way for new highways. Recently, some cities have reversed these trends by removing highways and establishing parks and public spaces. Two notable and beautiful examples: the Big Dig in Boston and the Embarcadero in San Francisco.

-Aesthetics. Many of these roads are about brute efficiency: moving the largest number of people in the shortest amount of time. To many, these highways scar the landscape. But they can often take on a beauty of their own, particularly in complicated interchanges.

-Small town life all but disappeared. With the rise of suburbs and highways rerouting traffic around small communities, rural populations dwindled.

-A fast-food approach to life. Not only does food have to be obtained quickly so one can get back on the road, signs need to be larger to be legible at 65 MPH, cars need to be larger to survive the occasional highway accident, travelers need built-in DVD players to be entertained, and so on.

Prior to the signing of this act, local governments and states had begun to cobble together a highway system. The City of Chicago had been planning for a local highway system for years but did not begin construction until after World War II. Pennsylvania had a turnpike (now I-76) and Indiana, Ohio, and Illinois had started roads that would create an interstate toll road. Robert Moses had begun a system in New York City.

But this law helped build and codify a system that is still going strong today.

Flying car cleared for take-off

Perhaps the predictions from the mid 20th century about flying cars may become reality. (Or maybe not.) Regardless, the Terrafugia Transition has been approved as a “light sport aircraft” by the Federal Aviation Administration. The Telegraph gives some of the specs:

The two-seater Transition can use its front-wheel drive on roads at ordinary highway speeds, with wings folded, at a respectable 30 miles per gallon. Once it has arrived at a suitable take-off spot – an airport, or adequately sized piece of flat private land – it can fold down the wings, engage its rear-facing propellor, and take off. The folding wings are electrically powered.

Its cruising speed in the air is 115mph, it has a range of 460 miles, and it can carry 450lb. It requires a 1,700-foot (one-third of a mile) runway to take off and can fit in a standard garage.

The aircraft/car is expected to sell for just under $200,000 so it’s not exactly ready for the mass market. There are some suburban aircraft communities – they typically have houses surroundings runways so pilots can taxi their small planes right to their garage. But those communities still have regular aircraft, not a plane you could fly to your workplace and then drive to Wal-Mart. And just imagine skipping an interstate traffic jam by taking off.

Final question: does the Transition fit through a standard fast-food drive-through lane?

Winning vs. a country’s culture

Brazilian coach Carlos Caetano Bledorn “Dunga” Verri has had a successful World Cup run thus far: four matches and four pretty easy wins. For many national coaches, this would lead to general praise from the media and fans.

But not in Brazil. Dunga has been playing with a more defensive-minded system, particularly compared to the attacking-with-flair Brazilian teams of past decades. A quick description of the battle Dunga has been fighting:

Then there were the fans, who almost always favor the spectacular and revel in the nation’s tradition of breathtaking open-field play.

Brazil has always been about offense, offense, offense. It has the deepest pool of talent in which to select a team. Its players pride themselves on creativity.

Even some of the  country’s former stars, such as Carlos Alberto, captain of Brazil’s 1970 team, have blasted Dunga:

“I am not confident in this group because our national team do not play Brazilian football…I’m talking about movement and use of the ball. We have good defenders, but the midfielders: if you ask Brazilian kids, who are our midfielders, they shrug their shoulders.”

So if Brazil wins a sixth World Cup title, what then? Will the country celebrate in the same way or will it be considered a less-than-great title? Sports fans can be an interesting lot, particularly when they are used to winning.

“The Triumphant Decline of the WASP”

A NY Times opinion piece from Harvard law professor Noah Feldman makes this argument: “The decline of the Protestant elite is actually its greatest triumph.” Feldman explores the changes in the Supreme Court (the appointment of Kagan would make it 6 Catholics, 3 Jews) and Princeton (“As late as 1958, the year of the “dirty bicker” in which Jews were conspicuously excluded from its eating clubs, Princeton could fairly have been seen as a redoubt of all-male Protestant privilege).

So what changed? Feldman provides some reasons: “the anti-aristocratic ideals of the Constitution,” education was an important defining trait for WASPs so opening up universities was a big step, and the American value of fair play. The result:

Together, these social beliefs in equality undercut the impulse toward exclusive privilege that every successful group indulges on occasion. A handful of exceptions for admission to societies, clubs and colleges — trivial in and of themselves — helped break down barriers more broadly. This was not just a case of an elite looking outside itself for rejuvenation: the inclusiveness of the last 50 years has been the product of sincerely held ideals put into action.

These may be accurate reasons. But they seem to ignore the historical context: something happened in the 1960s that changed institutions like Ivy League schools and led to a very different looking Supreme Court. In that decade, the Civil Rights Movement plus an explosion in higher education for the burgeoning US population plus higher rates of immigration from non-European locales plus cultural change (rock ‘n’ roll, television, more open questioning of authority, etc.), changed, or at least began to change, the socioeconomic status of WASPs.

Measuring celebrity

Forbes has released its annual list of the 100 most powerful celebrities. See Forbes’ website for a full portal that includes profiles of some of the celebrities and the full rankings (including rankings on subcategories). Topping the list is Oprah followed by Beyonce and James Cameron.

Buried at the bottom of the story is the methodology by which Forbes developed its list (the methodology is mentioned in this reposted story at Yahoo):

The Celebrity 100 is a measure of power based on money and fame. Earnings estimates, which include income from films, television shows, endorsements, books, and other entertainment ventures, are calculated between June 2009 and June 2010. Figures were rounded off where appropriate. Additional sources include Billboard, Pollstar, Adams Media Research, The Nielsen Company, and SNL Kagan. Fame is calculated using web hits on Google, Blog Search, TV/radio mentions on LexisNexis, overall press mentions on Factiva, and the number of times a celebrity’s image appeared on the cover of 25 consumer magazines. Social rank is calculated using metrics like Facebook friends and fans as well as Twitter followers.

I would be very interested in knowing the weights applied to each of these measures and broader categories (such as social rank). Take Lady Gaga for example: she is new to the list this year, does not have the media empires like some of the others on the list (Oprah’s big money advantage comes from an involvement in a multitude of media outlets), and yet benefits from a #1 ranking in the social rankings.

After a quick glance, money appears most important here. Perhaps having money prompts more media (of all kinds) mentions. Or perhaps the media mentions help build the money which then leads to a reinforcing cycle. Regardless, just having money may be a sign that you are a true celebrity. We as Americans may like our celebrities because they host a TV show or can do amazing things with a golf ball or can direct exciting movies, but just having money seems pretty interesting in itself.