Judging the validity of academic expertise in court

It is common in the world of academia for academics to judge the credibility of other scholars. But what happens when academics step into the courtroom and a judge assesses whether they are experts or not? Consider the case of a Canadian sociologist who was going to testify as a gangs expert:

Mark Totten, an Ottawa sociologist, has “virtually no expertise with gangs in the Greater Toronto Area,” Ontario Superior Court Justice Robert Clark said in a 27-page ruling which had been under a publication ban until the jury in a gang-related case began deliberations Wednesday.

Yet, this is the same sociologist who, in 2009, was praised by the Ontario Court of Appeal for having “extensive and impressive credentials” in the field of street gang culture…

Totten himself admitted in an interview that he “didn’t handle it very well” after wilting under cross-examination in the voir dire, a preliminary examination to determine the competency of a witness…

The last time Totten’s expertise was questioned was in 2007, when Justice Todd Archibald disallowed his “expert” witness testimony on the meaning of a teardrop tattoo on the cheek of an accused killer, Warren Abbey…

On his website, Totten’s list of degrees includes a PhD in sociology (1996) from Carleton University. He is also the author of a book about to be released, Nasty, Brutish and Short: The Lives of Gang Members in Canada.

According to his 31-page resumé, most of Totten’s work with gangs has been in the Ottawa area and western Canada, and he says he has counselled hundreds of gang members.

Several things seem to be happening here:

1. In the most recent incident, Totten admitted he didn’t do a good job testifying. So perhaps he isn’t convincing and/or gets flustered.

2. Perhaps Totten’s knowledge is not specific enough for particular cases. While he has researched gangs, he may not know the particulars of gang activity in Toronto (or some other locations).

3. With the possibility of #1 and #2, why would either the prosecution or defense call on Totten for his expert testimony?

4. How does a judge decide whether a testifying expert has enough expertise. I’m sure there are guidelines to this but doesn’t this require the judge to assess the research ability of the expert? For example, the recent case involved questions about the methodology Totten used:

In a ruling released March 5, Clark flagged as a “problem” Totten’s data relating to the sample size of gang members he purportedly interviewed, calling it “inaccurate and misleading in several ways.”

Clark had listened for a day and a half as Misener challenged Totten about his research and methodology, including that used in the Abbey trial, and about his lack of knowledge about Toronto street gangs.

This is a very common academic argument: attack the methodology of another researcher and suggest they can’t reach the conclusions they do because the data is bad. Knowing this, many academics know they have to be able to respond to this which is why articles and books typically contain a defense of the methodology used for the study. In this case, the argument seems to be that Totten can’t really speak about Toronto gangs because there are important differences between these gangs and the ones Totten has studied. At what point is the judge convinced that Totten is not an expert for this case?

Even if the methodology is good, perhaps #1 and #2 are most important here – if the expert can’t speak well to the specific case and defend their methodology, it doesn’t matter if the expert really is an expert. Part of being an expert requires that the expert can effectively communicate their argument and the methodology behind it.

(My goal in this post is not to defend Totten or suggest his testimony should not be allowed. Rather, I was intrigued by the fact that these arguments about methodology and validity took place in court. While sociologists and researchers in other disciplines might know how the publishing system works for their own field, I assume the rules and standards in court differ even as there are some similarities between the two realms.)

Going rogue

Wired’s Nate Anderson has a great write-up over at Ars Technica of the “Legitimate Sites v. Parasites” hearing before the U.S. House of Representatives Judiciary Committee today, and it’s not looking good for Internet intermediaries:

[T]he general mood of the hearing was that tough new steps must be taken. As Rep. Darrell Issa (R-CA) asked [Immigration and Customs Enforcement director John] Morton during his questioning, “What change in the law would allow you to pursue everyone?”

In his written testimony before the committee (PDF), Kent Walker, Google’s Senior VP and General Counsel noted that such an all-inclusive approach would be impossible and counterproductive:

When it comes to offshore rogue sites, no one should think that imposing additional obligations on search engines, social networks, directories, or bloggers beyond the DMCA [Digital Millennium Copyright Act] will be a panacea. If the site remains on the web, neither search engines nor social networks nor the numerous other intermediaries through which users post links can prevent Internet users from talking about, linking to, or referencing the existence of the site. These links or references will themselves appear in search results, and will enable users to reach the site. Simply put, search engines are not in a position to censor the entire Internet, deleting every mention of the existence of a site. If a rogue site remains accessible on the Internet, relying on search engines to try to make it “unfindable” is an impossible endeavor. [emphasis added]

I recommend reading Walker’s full comments for a robust defense of why the notice-and-takedown immunity provided by the DMCA is essential for innovation.

Additional coverage by Politico, Techdirt, CNET, TorrentFreak, RIAA Blog