False Negatives, False Positives, and Scholarly Journals

It is law review submission season, so I have had many conversations lately both on and off of social media about the student-run law review system. This recent poll by Professor Derek Muller was interesting, for example.

The quality of law review articles in top law journals is ___ it was ten years ago.

— Derek T. Muller (@derektmuller) February 22, 2024

I think it is a given that everybody thinks scholarly journals should publish better scholarship, and not publish worse scholarship. But as I thought about Professor Muller's question it seems to me that it's important to distinguish between the two different kinds of mistakes a journal can make: false positives (i.e., publishing something bad) and false negatives (i.e. declining to publish something good).

Given that no system is perfect, there is obviously some tension between these two goals. If you want to emphasize avoiding false positives then you should probably want a field in which there are a small number of highly-regarded journals with rigorous peer review on the basis of rigorous methods. Many potentially "good" pieces may get screened out of this system, but if something is published in the field's top journal you can basically take it to the bank.

On the other hand, if you want to emphasize avoiding false negatives then you should probably want a system closer to the current law review system, with many journals applying a much more pluralistic conception of merit, chasing pieces quickly through simultaneous submission. Most pieces that are significant and relevant can find a decent home.

Now neither system is perfect even at the goal it is trying to maximize -- there are rumors of corruption in even the most rigorously peer-reviewed fields, and there are still excellent pieces of legal scholarship that somehow don't fit the fashions of student editors. (And for that latter case may I again recommend the Journal of Legal Analysis, a peer-reviewed law review at Harvard Law School where I serve as a co-editor, especially for public law pieces?).

But I think it's helpful to articulate this distinction and these tradeoffs, and to remember that some changes that would fix one of these problems would make the other much worse. (For instance, returning to Muller's poll question, my hypothesis is that the law review system, taken as a whole, has perhaps gotten slightly better at avoiding false negatives (i.e. finding a home for good pieces) even if it has perhaps gotten slightly worse at avoiding false positives (i.e. letting bad pieces into good fora).

[For previous posts on law reviews, see here and here.]

UPDATE: An important and interesting corollary is the role of so-called specialty journals —the University of Chicago Business Law Review, the Harvard Journal of Law and Public Policy, the Yale Journal of Law and the Humanities, the Yale Journal of Law and Feminism, and so on. It seems to me that such journals are important for any areas where the flagship law reviews do have an unusually high number of false negatives.

For instance, one hears claims that the flagship law reviews do not accept (perhaps because they cannot adequately appreciate) even excellent pieces about tax law, private law, etc. Similarly, journals that focus on conservative thought, feminist thought, etc. may reflect a view that the flagship journals are insufficiently appreciative of good work with those attributes. These specialty journals provide another useful backstop against the false-negative problem in law review publishing.

At the same time, there might be specialty journals in areas that once had a great false-negative problem but where flagship journals as a whole have since started to self-correct. The Duke Journal of Gender Law and Policy, which ceased publishing in 2020 after 25 years may be an instructive example. In the Foreword to the final issue, Dean Kerry Abrams observed:

Unlike other journals that we have launched and then decommissioned, however, the gender journal has an additional, equally positive thread in its story . . . Gender, once a subject ignored or outright avoided by mainstream law journals, has now become a much more common subject for law reviews. In fact, in the last ten years, flagship law reviews have published the most influential and highly cited articles on gender and antidiscrimination law,[7] criminal law,[8] family law,[9] religion,[10] citizenship,[11] and abortion.[12]

7. See, e.g., Cary Franklin, The Anti-Stereotyping Principle in Constitutional Sex Discrimination Law, 85 N.Y.U. L. REV. 83 (2010).

8. Melissa Murray, Marriage as Punishment, 112 COLUM. L. REV. 1 (2012).

9. Clare Huntington, Postmarital Family Law: A Legal Structure for Nonmarital Families, 67 STAN. L. REV. 167 (2015).

10. Douglas Nejaime & Reva B. Siegel, Conscience Wars: Complicity-Based Conscience Claims in Religion and Politics, 124 YALE L.J. 2516 (2015).

11. See, e.g., Kristin A. Collins, Illegitimate Borders: Jus Sanguinis Citizenship and the Legal Construction of the Family, Race, and Nation, 123 YALE L.J. 2134 (2014).

12. Linda Greenhouse & Reva B. Siegel, Before (and After) Roe v. Wade: New Questions About Backlash, 120 YALE L.J. 2028 (2011).

If specialty journals are sufficiently successful in generating and demonstrating the importance of excellent scholarship in their domain, it is possible they will become less necessary to the next generation of scholars.

Overall, specialty journals have an important role to play especially in areas of scholarship with systematic false negatives.