In reading a book this weekend (a review to come later this week), I ran across a common tactic used by journalists: looking at the popularity of websites as evidence of a growing social trend. This particular book quoted a blog post and then said “The post got 110 comments.”
The problem is that this figure doesn’t really tell us much about anything.
1. These days, 110 comments on an Internet story is nothing. Controversial articles on major news websites regularly garner hundreds, if not thousands, of comments.
2. We don’t know who exactly was commenting on the story. Were these people who already agreed with what the author was writing? Was it friends and family?
In the end, citing these comments runs into the same problems that face web surveys done poorly: we don’t know whether they are representative of Americans as a whole or not. That doesn’t mean blogs and comments can’t be cited at all but we need to be very careful of what these sites tell us, what we can know from the comments, and who exactly they represent. A random sample of blog posts might help as would a more long-term study of responses to news articles and blog posts. But, simply saying that something is an important issue because a bunch of people were moved enough to comment online may not mean much of anything.
Thankfully, the author didn’t use this number of blog comments as their only source of evidence; it was part of a larger story with more conclusive data. However, it might simply be better to quote a blog post like this as an example of what is out on the Internet rather than try to follow it with some “hard” numbers.