While the ranking of undergraduate programs is contentious (read about Malcolm Gladwell’s latest thoughts on the subject here), the rankings of doctoral programs can also draw attention. In February, a five-person American Sociological Association (ASA) committee released a report about the 2010 National Research Council (NRC) rankings of doctoral sociological programs (see a summary here).
The ASA committee summarized their concerns about the NRC rankings:
Based on our work, we recommend that the ASA Council issue a resolution criticizing the 2010 NRC rankings for containing both
operationalization and implementation problems; discouraging faculty, students, and university administrators from using the core 2010 NRC rankings to evaluate sociology programs;
encouraging them to be suspicious of the raw data accompanying the 2010 NRC report; and indicating that alternative rankings, such as those based on surveys of departments’ reputations, have their own sets of biases.
The explanation of these issues is an interesting methodological analysis. Indeed, this document suggests a lot of these rankings have had issues, starting with the 1987 US News & World Report rankings which were primarily based on reputational rankings.
So what did the committee conclude should be done? Here are their final thoughts:
At this time, the committee believes that ASA should encourage prospective students, faculty, university administrators or others evaluating a given program to avoid blind reliance on
rankings that claim explicitly or implicitly to list departments from best to worst. The heterogeneity of the discipline suggests that evaluators should first determine what characteristics they value in a program and then employ available sources of information to assess the program’s performance. In addition, the ASA should help facilitate, within available means, the dissemination of such information.
So the final recommendation is to be skeptical about these rankings. This seems to be a fairly common approach for those who find issues with rankings of schools or programs.
How might we get past this kind of conclusion? If the ranking process were done by just sociologists, could we decide on even a fuzzy rank order of graduate programs that most could agree upon?