The ongoing mystery of counting website visitors

The headline says it all: “It’s 2015 – You’d Think We’d Have Figured Out How to Measure Web Traffic By Now.”

ComScore was one of the first businesses to take the approach Nielsen uses for TV and apply it to the Web. Nielsen comes up with TV ratings by tracking the viewing habits of its panel — those Nielsen families — and taking them as stand-ins for the population at large. Sometimes they track people with boxes that report what people watch; sometimes they mail them TV-watching diaries to fill out. ComScore gets people to install the comScore tracker onto their computers and then does the same thing.

Nielsen gets by with a panel of about 50,000 people as stand-ins for the entire American TV market. ComScore uses a panel of about 225,000 people4 to create their monthly Media Metrix numbers, Chasin said — the numbers have to be much higher because Internet usage is so much more particular to each user. The results are just estimates, but at least comScore knows basic demographic data about the people on its panel, and, crucial in the cookie economy, knows that they are actually people.5

As Chasin noted, though, the game has changed. Mobile users are more difficult to wrangle into statistically significant panels for a basic technical reason: Mobile apps don’t continue running at full capacity in the background when not in use, so comScore can’t collect the constant usage data that it relies on for its PC panel. So when more and more users started going mobile, comScore decided to mix things up…

Each measurement company comes up with different numbers each month, because they all have different proprietary models, and the data gets more tenuous when they start to break it out into age brackets or household income or spending habits, almost all of which is user-reported. (And I can’t be the only person who intentionally lies, extravagantly, on every online survey that I come across.)…

And that’s assuming that real people are even visiting your site in the first place. A study published this year by a Web security company found that bots make up 56 percent of all traffic for larger websites, and up to 80 percent of all traffic for the mom-and-pop blogs out there. More than half of those bots are “good” bots, like the crawlers that Google uses to generate its search rankings, and are discounted from traffic number reports. But the rest are “bad” bots, many of which are designed to register as human users — that same report found that 22 percent of Web traffic was made up of these “impersonator” bots.

This is an interesting data problem to solve with multiple interested parties from measurement firms, website owners, people who create search engines, and perhaps, most important of all, advertisers who want to quantify exactly which advertisements are seen and by whom. And the goalposts keep moving: new technologies like mobile devices change how visits are tracked and measured.

How long until we get an official number from the reputable organization? Could some of these measurement groups and techniques merge – consolidation to cut costs seems to be popular in the business world these days. In the end, it might not be good measurement that wins out but rather which companies can throw their weight around most effectively to eliminate their competition.

Use Airbnb to try a neighborhood before you buy a home

The neighborhood is an important part of purchasing a new home so I’m surprised it has taken so long to get to a solution like this: use Airbnb to try out the neighborhood before you buy.

Realtor.com and Airbnb have teamed up to show visitors to the realty website what Airbnb rentals are near properties listed for sale, so potential buyers can test-drive a neighborhood.

“This collaboration with Airbnb reinforces our commitment to giving consumers unparalleled insight to make informed real estate decisions,” Ryan O’Hara, chief executive officer of Realtor.com, said in a statement Wednesday. “Our relationship with Airbnb … allows us to reduce some of the unknown factors associated with relocating to a new community.”

I wonder how many people will take advantage of this. Even though Airbnb might make it easier to try the neighborhood yourself, it still requires the effort of signing up, actually staying, and then looking around and/or talking to people. And if you are going to go to the trouble to walk around and talk to people, do you actually need to stay the night? Remember, many neighborhood members may just be trying to avoid each other (examples here and here).

Perhaps the next step in all of this is to find a way for people to stay at the prospective home itself before they buy. Perhaps you could get two days and one night to stay there but have to put down a hefty fee. This may not work if the homeowners are still living there but it would offer an unparalleled look at a major purchase.

Mark Zuckerberg encouraging people to read sociological material

Mark Zuckerberg has been recommending an important every two weeks in 2015 and his list thus far includes a number of works that touch on sociological material:

Zuckerberg’s book club, A Year of Books, has focused on big ideas that influence society and business. His selections so far have been mostly contemporary, but for his eleventh pick he’s chosen “The Muqaddimah,” written in 1377 by the Islamic historian Ibn Khaldun…

Ibn Khaldun’s revolutionary scientific approach to history has established him as one of the foundational thinkers of modern sociology and historiography…

The majority of Zuckerberg’s book club selections have been explorations of issues through a sociological lens, so it makes sense that he is now reading the book that helped create the field.

A Year of Books so far:

  • “The End of Power: From Boardrooms to Battlefields and Churches to States, Why Being In Charge Isn’?t What It Used to Be” by Moisés Naím
  • “The Better Angels of Our Nature: Why Violence Has Declined” by Steven Pinker
  • “Gang Leader for a Day: A Rogue Sociologist Takes to the Streets” by Sud hir Venkatesh
  • “On Immunity: An Inoculation” by Eula Biss
  • “Creativity, Inc.: Overcoming the Unseen Forces That Stand in the Way of True Inspiration” by Ed Catmull and Amy Wallace
  • “The Structure of Scientific Revolutions” by Thomas S. Kuhn
  • “Rational Ritual: Culture, Coordination, and Common Knowledge” by Michael Chwe
  • “Dealing with China: An Insider Unmasks the New Economic Superpower” by Henry M. Paulson
  • “Orwell’s Revenge: The 1984 Palimpsest” by Peter Huber
  • “The New Jim Crow: Mass Incarceration in the Age of Colorblindness” by Michelle Alexander
  • “The Muqaddimah” by Ibn Khaldun

An interesting set of selections. At the least, it suggests Zuckerberg is broadly interested in social issues and not just the success of Facebook (whether through gaining users or producing sky-high profits). More optimistically, perhaps Zuckerberg has a sociological perspective and can take a broader view of society. This could be very helpful given that his company is a sociological experiment in the making – not the first social networking site but certainly a very influential one that has helped pioneer new kinds of interactions as well as changed behaviors from news gathering to impression management.

The more cynical take here is that this book list is itself an impression management tool intended to bolster his reputation. Look, I do really want the best for our users and society! However, would this be the set of books that would most impress the public or investors? Listing sociology books as well as books regarding sociological topics may only impress some.

Autonomous vehicles to intensify motion sickness

A new study suggests self-driving cars will make motion sickness worse:

For adults, motion sickness will be more of an issue in self-driving vehicles than in conventional vehicles. Some are expected to experience motion sickness often, while others may actually feel sick every time they’re riding in an autonomous vehicle, a study by researchers at The University of Michigan’s Transportation Research Institute revealed…

Mr. Sivak and his co-researcher Brandon Schoettle looked at the three main factors that cause motion sickness (conflict between vestibular and visual inputs; inability to anticipate the direction of motion; and lack of control over the direction of motion) and determined that they are elevated in self-driving vehicles…

It’s become evident that self-driving cars will replace traditional cars in the future, and when this happens, all adults (who are most prone to motion sickness) will be passengers at all times. Mr. Sivak clarified that being a passenger in an autonomous vehicle will be quite different than riding along in a train or other mode of public transportation, for, unlike trains, self-driving cars will be subject to more lateral acceleration/deceleration as well as longitudinal acceleration/deceleration that is drastically less smooth. The small windows won’t help either.

The other major factor in the increased prevalence of motion sickness is what adults will do whilst in cars instead of driving. In an opinion survey of 3,255 adults from the U.S., China, India, Japan, Australia and the U.K., respondents named reading, talking/texting, sleeping, watching movies/TV, working and playing games as the activities they’ll engage in while riding in self-driving cars. According to the study, almost all of the activities mentioned worsen the frequency and severity of motion sickness.

An interesting side effect of a new technology. But, this means that automakers can/should include virtual reality devices to ease the ride – they won’t just be cars but rather entertainment pods! Everyone can be like the kids of today who ride in the expensive minivans and SUVs watching their split screen entertainment systems while holding their tablets and smartphones…

“Why Is My Smart Home So Stupid?”

A marketing professor gives an answer to this simple question:

One popular answer is that the Internet of Things is still in its infancy and that better technology and standards are within reach and will lead to greater integration, and thus, greater smartness in the not too distant future.

There is some value in this explanation. Everyone who has ever tried to get an IP camera to work on a cell phone will probably agree. But this answer is also entirely steeped in a technological mindset and the naive belief that better technology will automatically improve our lives.

An alternative explanation may be that popular tropes such as the “Internet of Things” not only inspire but also constrain our imagination as innovators and as consumers. Designing greater customer experiences and, thus, extracting greater economic value may be a matter of avoiding this trope altogether…

One managerial implication we can derive from Epp, Schau, and Price is that different smart home definitions are possible. And Nest’s definition seems much more powerful than Plum’s. Plum adds yet another layer to the Internet of Things, and the result is often a home where everything is connected but nothing adds up. In sharp contrast, Nest succeeds by putting its technology in service of a much higher sociological goal: the age-old quest to create and sustain a happy family

The suggestion here is that new technology is only as good as the improvements in social interactions that it brings. Way before the smart home, modern consumers have been promised all sorts of benefits from new technology but the created items don’t always lead to the desired social outcomes. Cars enabled easier transportation but led to more private existences and increasing sprawl. Similarly, more single-family homes gave people space but helped spread them out. The radio and later television delivered mass media, theoretically connecting people, but also led to people sitting around these items. Modern appliances were to save labor. The Internet allows unprecedented customized access to information yet can lead to echo chambers and isolated interactions. Autonomous vehicles will create more free time or more time to work?

Perhaps this should be a challenge for smart home innovators: how can new devices both help in their particular area (say heating or lighting or saving energy) and foster social interaction? This may actually be the harder part.

How to pronounce McMansion (courtesy of YouTube)

The Internet might bring some wonderful things but it can also make you scratch your head. Here is a YouTube video for how to pronounce McMansion. The video doesn’t exactly have a lot of views – 1 after I watched it! – and comes courtesy of DictionaryVoice.com.

Two quick thoughts:

1. I admit that I have looked up pronunciations through online dictionaries and had the site read to me. This can be a very handy Internet tool.

2. The next video YouTube plays after this one is the song “Jesusland” from Ben Folds. This is one of the few pop songs I know that mention McMansions and, fitting the common use of the word, Folds uses it as part of this critique of Middle America. Here is the portion of the song where it comes up:

Down the tracks, beautiful McMansions on a hill
That overlook a highway with riverboat casinos
And you still have yet to see a soul

Not too different from those depictions in Gone Girl…

Predicting riots using social media

In addition to the identified factors from research coming out of the 1960s and 1970s, one sociologist suggests social media activity can show how riots and protests spread:

The most promising method of “predicting” unrest might be through social media. Dan Braha, a professor at the University of Massachusetts and affiliate of the New England Complex Systems Institute, has studied unrest in hundreds of countries and the phenomenon of “contagion,” or how it spreads. In the past, printed newspapers, televisions, and other media played an important role, he said. “Today, the use of Twitter, Facebook, and other social media platforms is fundamental to the rapid self-organization and spreading of unrest activities—much like the spread of fire in a forest.” And the data from these media can be tracked. Riots, he claims, are certainly foreseeable, but “prediction regarding ‘when’ and ‘where’ becomes more precise on short time scales.”

It sounds like social media is just part of the puzzle here. There are certain underlying conditions mentioned in this article – such as hot weather or precipitating incidents (such as police violence) – but these do not always lead to riots. (In fact, given the inequalities present in many American cities, riots and protests could be considered relatively rare.)  Just as with the analysis of the Arab Spring activity, social media does not cause protests or riots but it can help facilitate it. This was reported in Egypt as protestors shared information through social media and even peer-to-peer options. This was also reported in Baltimore as protestors selected places to show up. This is not a new phenomena; riots in the 1960s spread in a contagion like manner and the dispersion could be tracked through news coverage in the New York Times. But, the availability of social media now makes it theoretically possible to watch things develop in real time, an advantage for both protestors and authorities.

Thinking to the future, what happens when protestors make use of non-public social media or peer-to-peer options that cannot be viewed by authorities?

Social scientists critique Facebook’s study claiming the news feed algorithm doesn’t lead to a filter bubble

Several social scientists have some concerns about Facebook’s recent findings that its news feed algorithm is less important than the choices of individual users in limiting what they see to what they already agree with:

But even that’s [sample size] not the biggest problem, Jurgenson and others say. The biggest issue is that the Facebook study pretends that individuals choosing to limit their exposure to different topics is a completely separate thing from the Facebook algorithm doing so. The study makes it seem like the two are disconnected and can be compared to each other on some kind of equal basis. But in reality, says Jurgenson, the latter exaggerates the former, because personal choices are what the algorithmic filtering is ultimately based on:“Individual users choosing news they agree with and Facebook’s algorithm providing what those individuals already agree with is not either-or but additive. That people seek that which they agree with is a pretty well-established social-psychological trend… what’s important is the finding that [the newsfeed] algorithm exacerbates and furthers this filter bubble.”

Sociologist and social-media expert Zeynep Tufekci points out in a post on Medium that trying to separate and compare these two things represents the worst “apples to oranges comparison I’ve seen recently,” since the two things that Facebook is pretending are unrelated have significant cumulative effects, and in fact are tied directly to each other. In other words, Facebook’s algorithmic filter magnifies the already human tendency to avoid news or opinions that we don’t agree with…

Christian Sandvig, an associate professor at the University of Michigan, calls the Facebook research the “not our fault” study, since it is clearly designed to absolve the social network of blame for people not being exposed to contrary news and opinion. In addition to the framing of the research — which tries to claim that being exposed to differing opinions isn’t necessarily a positive thing for society — the conclusion that user choice is the big problem just doesn’t ring true, says Sandvig (who has written a paper about the biased nature of Facebook’s algorithm).

Research on political echo chambers has grown in recent years and has included examinations of blogs and TV news channels. Is Facebook “bad” if it follows the pattern of reinforcing boundaries? While it may not be surprising if it does, I’m reminded of what I’ve read about Mark Zuckerberg’s intentions for what Facebook would do: bring people together in ways that wouldn’t happen otherwise. So, if Facebook itself has the goal of crossing traditional boundaries, which are usually limited by homophily (people choosing to associate with people largely like themselves) and protecting the in-group against out-group interlopers, then does this mean the company is not meeting its intended goals? I just took a user survey from them recently that didn’t include much about crossing boundaries and instead asked about things like having fun, being satisfied with the Facebook experience, and whether I was satisfied with the number of my friends.

New gadgets, apps want more location data from users

Location data is valuable and more new gadgets make use of the information:

Location-tracking lets developers build fast, useful, personalized apps. They’re enticing, but they come with tradeoffs: your gadgets and apps maintain a log of where you’ve been and what you’re doing, and more of them than you think are sharing that data with others.

It’s going to advertisers, mostly, so they can lure you into the Starbucks a block away or the merch tent at Coachella. It’s as creepy as any other targeted marketing, but most of us have come to accept that it comes with the territory. Jennifer Lynch, a senior staff attorney at the Electronic Frontier Foundation, says it goes deeper. Your data might get sold to your credit reporting agency, which wants to know more about you as it determines your credit score. It might go to your insurance company, which is very interested in your whereabouts. It might be subpoenaed by the government, for just about any reason. Maybe none of that is happening. Maybe all of it is. There’s really no way for us to know…

Your phone’s ability to pinpoint your exact location and use that info to deliver services—a meal, a ride, a tip, a coupon—is reason for excitement. But this world of always-on GPS raises questions about what happens to our data. How much privacy are we willing to surrender? What can these services learn about our activities? What keeps detailed maps of our lives from being sold to the highest bidder? These have been issues as long as we’ve had cellphones, but they are more pressing than ever.

Another major trade-off that I suspect most users will make without much fuss in the coming years. The cynical take on the advantages for the user is that this is primarily about customizable marketing that can account for both your individual traits and where exactly you are. In other words, sharing location data will give consumers new opportunities. More consumerism! On the flip side, it is less clear how or when location data might be used against you. But, when it is, it probably won’t be good.

The broader issue here is whether people should have geographical freedom that is not known to others. This is increasingly difficult in today’s world even as we would celebrate the mobility Americans have within their own communities, country, and to travel throughout the world.

What will we all do in autonomous cars? Use 4G, wireless connections

With the rapidly approaching autonomous vehicles, how will we spend the time once devoted to driving (or backseat driving)?

According to the U.S. Census Bureau, approximately 105 million Americans commute by car each day. With an average round trip of 50 minutes, that’s a whopping 88 million hours daily. “I want some of these hours back,” said Cooley. At this point, “we’re now already moving into the era of post-self-driving vision. It’s not so much about the technology of self-driving, but what will we do after that.”…

The most important groundwork, or “plumbing,” as Cooley refers to it, is for auto manufacturers to integrate 4G into vehicles. Built-in high-speed wireless connectivity makes a car much like a phone, Cooley said, allowing users to get into the habit of using maps, calls, notifications, and other interactive elements in the car.

Given today’s world, this isn’t too surprising: cars will free up even more time for Internet and social media usage. Indeed, we’ll have extra time for multitasking where we can listen to music or watch something while participating online (while being a passenger). Are there better things we could do? With all the studies on sleep deprivation (partly due to media usage), perhaps not driving should lead to more sleep. Or, employers and workers might do more job-related activities on the way to and from work.

Alas, all of these supposed time-saving devices may just keep providing opportunities to do more work or entertain ourselves further…