Has the Supreme Court really moved leftward this term?

(J. Scott Applewhite/AP)

Earlier this week the New York Times proclaimed that the Supreme Court has "move(d) leftward." Much like earlier pronouncements that the Roberts Court was the most conservative in decades—particularly those based upon similar types of analyses, the article's central claim needs to be taken with healthy dose of salt.

The central claim of the article is that the Supreme Court has had one of its most "liberal" terms since the end of the Warren Court. This conclusion is based upon an analysis which finds that the Court has adopted a "liberal" outcome in 54 percent of cases that have been decided thus far this term. Such an analysis, combined with consideration of the frequency each justice finds him or herself in the majority, may tell us which "side" of the court is prevailing more often this term, but I do not think it tells us all that much about the trajectory or tendency of the Court or its jurisprudence.

One immediate problem with this sort of analysis is that it does not account for the substance of individual cases and, more importantly, the effect on underlying doctrine. That is, this sort of analysis makes no distinction between a case that shifts the law in a more conservative or liberal direction and a decision that maintains the status quo. Assuming that, at least in some areas, the current justices are relatively satisfied with current doctrine, whether a case is coded as "liberal" or "conservative" will be solely a function of the judgment under review.

Another problem with this sort of analysis, particularly when used to analyze the Court's behavior over time, is that it does not account for shifts in the law. As a consequence, this sort of analysis can produce conclusions that are precisely the opposite of what is actually occurring. That is, if a Court adopts a liberal holding at one point in time, and then refuses to extend that holding still further in the latter, the decision maintaining the comparatively liberal rule will be coded as a "conservative" holding, even though all it did is maintain the status quo.

To illustrate this problem, consider the Court's global warming decisions. In Massachusetts v. EPA, in 2005, the Court held that the greenhouse gases were "pollutants" subject to regulation under the Clean Air Act. This holding dramatically expanded the EPA's regulatory authority. The decision also lowered the bar for standing in environmental cases for state litigants, if not more generally. This was clearly a "liberal" ruling. Several years later, in American Electric Power v. Connecticut, the Court held that nuisance suits against greenhouse gas emitters under federal common law were displaced by federal regulatory authority. This holding was a direct consequence of the Massachusetts v. EPA holding, and would have been classified as a "conservative" ruling.

Taken together, Mass. and AEP would represent a wash—a liberal decision and a conservative decision. Yet, as a substantive matter, the combination is a dramatic shift in the law in a "liberal" direction. Analyzing individual votes only magnifies the problem, as there were 12 votes for the "conservative" position in the cases combined, while only five votes for the "liberal" position. The sort of analysis embodied in the Times article would suggest that the Mass. Court was more liberal than the AEP Court, while a consideration of the actual decisions would find no change at all. AEPdid nothing at all to scale back the holding of Massachusetts v. EPA, so the consequence of this "more conservative" Court was nothing but a maintenance of the status quo.

Another concern I have with the article is its uncritical acceptance of the case coding in the Supreme Court Database. While conceding that it is "possible to quarrel with the coding of any individual case," the article's authors claim that "there is relatively little disagreement about the judgments among legal scholars, and the coding conventions are both consistently applied and in line with most people's intuitions." This may be the view of most political scientists, but it is hardly a consensus view. Indeed, multiple analyses of the Supreme Court Database's case coding have found widespread instances of questionable coding, affecting as much as 20 to 30 percent of cases. See, for instance, the work of Carolyn Shapiro here and here. Indeed, the problems are bad enough that some scholars who heavily relied upon the database in the past have recoded cases for more recent analyses (such as this one, which was also the subject of a Times story and discussed here).

While the coding of most contemporary cases is unlikely to be controversial (save for those where the Court splits along untraditional lines), the lack of consistent or reliable coding in the database limits the usefulness of historical analyses. This is particularly so given evidence that some coding may have been the result of confirmation bias, such that coding of some cases may have reflected coder expectations about the Court's behavior in a given term as much as the actual merits of the case.

Another concern I have, acknowledged in the piece, is the focus on a single term. Given the relatively small number of cases the Court hears each term, no single term is particularly representative of the Court's work as a whole. Thus it is inevitable that some terms appear more "liberal" or "conservative" than others because few, if any, terms contain a fully representative sample of the sorts of issues the Court is called upon to address. This term, for example, despite the heavy roster of high profile cases, still did not include cases in many areas that regularly divide the justices along ideological lines, such as abortion.

If we really want to know whether the Court is more liberal or conservative than it has been in the past—whether in terms of its trajectory or its performance in a given term—we need to do more than code each case as "liberal" or "conservative" and tabulate the results. Instead we must look at the substance of the Court's decisions to see whether it is moving the law in one direction or another—whether expanding gay rights and limiting the death penalty or constraining federal power and reducing protections for criminal defendants.

In the past I've argued that a substantive analysis of the Roberts Court suggests that it is a generally a "conservative minimalist" court. That is, the modal behavior of this Court is to move the law slowly, but perceptibly, in a rightward direction, while maintaining a fairly heavy status-quo bias. There are exceptions, however, as there are areas in which the Court's shift have not been minor (the protection of campaign-related speech) and still others where the Court has moved the law in a more liberal direction (gay rights and habeas rights for detainees). Further, in some areas in which the Court has shifted Right, such as abortion, it appears to have brought us back to where the Court had been in the early years of the Rehnquist Court. The Court may be more conservative than the Warren and Burger Courts, but it is doing very little to undo most Warren and Burger Court precedents.