Maybe the Status Quo Isn't So Bad? Thoughts on Two Proposals about Law Review Reform from the AALS

Sorting at scale is always going to be hard.

|The Volokh Conspiracy |

Brian Galle, writing on behalf of an Advisory Committee on Law Journal Reform of the AALS Section on Scholarship, has posted two proposals to reform the law review submission process.  The proposals are very interesting, but I'm skeptical they would work.  On the whole, I suspect they would do more harm than good.  In this post, I will explain why.

Let me start with an overview of the proposals. According to the recently-posted discussion draft, the main problem with the law review submission process is that its scale makes for poorly-informed decisions.  In the current system, there are hundreds of general-interest law reviews and thousands of law review authors. This means that a single journal can receive several thousand submissions.  Editors can't possibly make informed decisions about them in the time-crunch of submission season.  And in the crunch of decision-making time, journals typically receive hundreds of expedite requests requiring them to make very quick decisions about whether to accept articles.  They can't make decisions in an informed way, so they rely too often on proxies and just can't make the thoughtful evaluations of scholarship that would be best.

The authors offer two proposals for reform.

The first proposal is to mandate limits on submissions and acceptances.  Authors would be forced to limit how many submissions they make at any one time (say 10 or 20), and they would be required to accept the first offer they receive.  Also, journals would be forbidden from making offers in a short period of time after submission (say, a month).  The result, according to the idea's proponents, would be a more orderly system of submissions and acceptances.  The rules to implement this proposal would be promulgated by a selected committee of law professors called the "Selection Committee."  The Selection Committee would police violations of its rules and could punish violators with sanctions, such as temporarily forbidding access to the submission system.

The second proposal is a matching system. Every author would choose a set of journals in which they would be willing to publish.  Every journal would go through all the submissions and decide which ones they are willing to publish. A computer would then match article to journal based on the mutually expressed preferences.  The results would be binding on both authors and journals.  The matching system would be run by the Selection Committee, as above.

I appreciate the group's very thoughtful engagement with hard design questions.  But I'm skeptical that either of these proposals would be an improvement.  My initial reaction is that both proposals would just make the system worse.

Here's my thinking.

Let me start with what I take to be the big problem here.  It's a problem that the proposal hints at, but as far as I can tell doesn't actually identify: The prominence of the general-interest law review.  For quirky historical reasons, most of the major law reviews are general.  They will consider any article that remotely touches on anything relating to law.  An article could cover any field of law, whether it is constitutional law, comparative law, tax law, jurisprudence or any other legal field.  An article could be theoretical, doctrinal, or empirical.  And the connection to law can be (and often is) modest.  An article could really be about history, or economics, or psychology, or sociology, or any other field as it touches on some aspect of law.  Everything under the sun can be considered by any general-interest law review.

As I see it, the general-interest law review is the real source of the problem the new proposal identifies.  Several thousand law-related submissions are created every year. If you're in charge of a general-interest law review, you're open to considering every single one of those several thousand submissions.  And if most of the major journals are general-interest law reviews, that means that most of the journals are open to considering every single one of those submissions.  And because all of those general-interest law reviews are offering essentially the same service, they compete for author interest based primarily on prestige.  In a world in which submitting an article is relatively inexpensive, authors have an incentive to submit widely to find the most prestigious journal that will accept their papers.

In short, the prominence of the general-interest law review makes the scale of the problem inherently unmanageable.  From the journal side, we have hundreds of journals competing for the best articles they can get from thousands of submissions on every legal topic under the sun.  And from the author side, we have thousands of authors competing for the best placement they can from hundreds of journals.  If you want an orderly system in which decisions are made slowly and deliberately, figuring out a way to match up that many articles with that many spots in journals is an incredible challenge. (Professors often complain that student editors don't do a good job selecting papers, but I doubt professors could do much better faced with these sorts of numbers.)

If that's the problem, what's the answer?

One obvious answer would be to simply abolish general-interest law reviews.  We could instead have a siloed system like we normally have in other academic fields.  Each journal would have a specific subject matter or methodology.  For example, instead of the Harvard Law Review, you could have the Harvard-based Journal of Constitutional Theory.  Instead of the Yale Law Journal, you could have the Yale-based Jurisprudence Review.  Instead of the Stanford Law Review, you could have the Stanford-located Papers in Intellectual Property Law.  Instead of the Columbia Law Review, you could have a Columbia-based Business Law Review.  (Of course, there are many subject-based law reviews today, including, as it turns out, a Columbia Business Law Review. But my sense is that more of the placement angst surrounds the general-interest law reviews.)

If scale is the real problem, that strikes me as the real way to solve it.  If every journal picked a subject area or methodology, each would only consider a limited subset of articles.  This narrowing would mean that authors would have only a handful of journals that would even consider their papers.  Authors would submit to only those journals, and would accept at the best of them or not publish the article if they receive no offers.  Each journal would get a limited number of submissions simply because they would not consider submissions outside their narrow field.  The scale problem would be solved.

Am I actually recommending that general-interest law reviews should be abolished?  No. Although this would solve the scale problem, I think it would also eliminate some of the major strengths of the status quo.  For example, the prominence of the general-interest law review creates an incentive to write in a more accesssible way.  General-interest law reviews also circumvent the gate-keeping function of subject matter silos that I suspect would block new ideas from entering the academy.  So I don't actually want them gone. My point is just that the scale problem the discussion draft tries to solve is unavoidable when you have so many general-interest journals being open to publishing so many submissions.  The discussion draft doesn't question the prominence of general-interest law reviews, so it necessarily offers proposals to deal with the crazy scale of the matching problem rather than to change it.

Let's turn to the two specific proposals.  Would they make the probem better or worse?  I suspect they would make the problem worse.  Here's why.

The first proposal, limiting submissions and acceptances, strikes me as an overly heavy-handed way to limit choice.   Limiting the number of submissions will, as a practical matter, force authors to eliminate the journals that they think are unlikely to accept their articles.  I worry that authors would limit themselves by foregoing the opportunities of getting more prestigious placements, which will have the effect of deepening existing hierarchies.  The lost opportunity for authors would be matched by the lost opportunity for journals, preventing the matching that is the current system's main strength,  And requiring journals to wait a month before accepting an article will draw out the process for too long.  Maybe I am too much of a libertarian.  But I think it's better both for authors and for journals to have a greater set of choices.

The second proposal, a complete ranking system, strikes me as impractical.  If it could work, it sounds like a good answer:  Everyone's preference is optimized.  But how is any journal supposed to rank several thousand submissions?  And how is every journal supposed to do that?  It's like making a committee of law students grade thousands of 60-page papers.  And it's worse than that: Journals not only would have to grade the papers, but they have to put each of them in exact order of preference (up to some quality standard they need to identify, if they can do so in the abstract).  I've been on committees of professors where we tried to rank a handful of law review articles in one field, and it was highly contested and difficult.  How can we expect law students to do that for hundreds or even thousands of articles from every field at once?  If I am understanding the proposal correctly, it doesn't seem like something that editors can do.

If I had to pick one of the three options—the status quo, the first proposal, or the second proposal—I would favor the status quo.   The current selection process is messy and imperfect.  But it also has considerable strengths that are too-often overlooked.  Editors can look through submissions for diamonds in the rough.  The expedite process can focus journals on a subset of admissions that have been judged by peers as worthy of consideration.  Granted, it's understandably frustrating for editors when a higher-ranked journal picks off an article they found.  But it's a frustration that some amount of market sorting is going to produce (and my sense is that editors frustrated by losing to higher-ranked competitors have less of a problem with picking off articles from journals ranked below them).  Finally, because journals are competing primarily on prestige, the stakes of journals measuring quality incorrectly are relatively low.

As I said above, the status quo is imperfect.  But I see a lot of that imperfection as the fault of the general-interest law review.  In a world with hundreds of general-interest law reviews and thousands of eligible articles, the matching process is destined to be difficult. If we're not going to abolish general-interest law reviews, we need to pick which set of serious problems with article selection are less troublesome than others.  And my own sense is that the status quo is likely less troublesome than the particular alternatives discussed by the AALS Advisory Committee.

 

 

 

NEXT: There Is No Line

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  1. The whole concept of submitting an article to multiple journals has always seemed very odd to me. Elsewhere in the sciences (where a substantial amount of peer review effort is put into every paper) authors are required to pick a single journal to submit their article to.

    1. Second this.

      In engineering it’s considered unethical to have the same manuscript under review at two different journals.

      1. YWhen I was in engineering graduate school, most journals were extremely specific about their subject matter. If you had a paper in computational fluid mechanics, you submitted it to one of a handful of journals specifically about computational fluid mechanics. To make the comparison more direct, you’d have to imagine how the system would work if each university had a main journal that was about all science, or perhaps all engineering, and that received thousands of submissions. It would be a lot harder to have a single submission system: If it takes 6 months to be rejected, and a journal takes 10 articles out of 5,000 submissions, it could take authors several centuries before their articles find a home.

        1. I don’t know the precise rules in finance, but I do know it may be years between submission and publication, what with peer review, revision and resubmission, and so on.

          It sounds like any grammatical law review submission is going to get published someplace, and the problem for the system is figuring out where.

  2. Editors can’t possibly make informed decisions about them in the time-crunch of submission season.

    And yet, to all appearances, they do. The vast majority of submissions are quick rejects, just like the vast majority of cert. petitions in the Supreme Court. Many are obviously bad, many are too narrow for a general-interest law review, many will require too much work to become publishable, many are just not interesting.
    That said, it does strike me, as an original matter, as rather silly for students to be evaluating the work of seasoned (or up-and-coming) scholars, but the proxies they rely on to compensate for their lack of expertise seem to work. I have yet to see anything suggesting that valuable scholarship is not being published, and nothing seems likely — given the number of publication outlets — to prevent the publication of poor stuff.
    Is all of this a solution in search of a problem?

  3. It’s nice to see thoughtful proposals and thoughtful discussion on any topic in today’s world.

    But it confirms to me the one thing that lawyers seem incapable of doing — being brief. I have always had trouble with insomnia. My cure it to listen in bed to podcasts of lawyer discussions. They go on and on and bore me to sleep.

    This brief little blog post was 1784 words, 150% of a TED talk.

    1. anorlunda, thanks for your comment. I rarely blog because most of my public writing is at Twitter, which limits comments to 280 characters. As my bio states, you can find me there at twitter.com/orinkerr

  4. When I was a law review editor we had no problem separating the responsibly written submissions from the crackpots. Since then a lot of the crackpot ideas have become law.

  5. The bigger question is why have law reviews at all? There is no pre-publication peer review like there would be in other scholarly journals. For the most part, the student editors just copyedit the text and check the cites.

    Why not just set up an internet “warehouse” where anybody can upload their articles, and make access simple (i.e. a good search engine and non-anonymous reviews)

  6. I’m not certain what problem we’re solving for but I have a different perspective. I’m an editor of a journal with a specific subject matter and regional focus. The articles’ authors historically have not been academics or students at all. They’ve been practitioners in the field. The journal does not have a lot of notoriety or prestige (however defined) so far as I can tell, and I would be surprised if academics or students were aware of it at all. The audience for the journal is almost exclusively practitioners within a specific section of my state’s bar.

    Since no one knows about the journal, we don’t receive submissions the way a prominent general interest law review or journal might. Accordingly all the topics have to be selected by the editorial board (made up entirely of practitioners–no students or “academics” although several of the board members are adjunct professors) and then authors assigned and/or solicited from the section’s ranks.

    I would love to raise the profile of the journal so we received submissions. Very good articles on topics directly germane to our journal’s specific practice and regional interest have been published in the past few years… in general interest law reviews. And I don’t even know how to begin the process of competing for those articles.

    General interest law reviews have something to sell that my journal doesn’t. First, a lot of them have rivers of history that I can’t replicate overnight. Second, I’m working from a sub-specialty that is no darling to the academia, and so the smartest people with the most time on their hands for publishing–professors–probably wouldn’t want to write in our journal. And even if it’s a topic appropriate for our journal, what they want (massive attention) I can’t give them anyway, because I don’t have the prestige yet. And I doubt we’ll ever have the prestige, because part of what makes the Most Prestigious Law Review so great is its already sterling reputation for issuing thoughtful papers on many different topics. Prestige is basically defined by citations, and general interest are going to get more citation because they have more output compared to special-interest journals, full stop.

    I’d be interested to know if you submitted to the CLSR or its equivalent when you wrote Internet Surveillance Law after the USA Patriot Act and A User’s Guide to the SCA. I don’t know how to get the Professor Kerr’s of the world to write into my sub-specialty journal, but I’d love to find out.

  7. These proposals sounds like something that drives me nuts: the authors saw what they thought was a similar problem elsewhere and copied the solution without understanding what it was.

    To wit: this is almost the exact model used in the medical school to hospital pipeline. In that model soon-to-be doctors apply to dozens or hundreds of hospitals, including face to face interviews, and the hospitals then indicate which of the ones they want. Then the graduating students are assigned to their highest priority hospital that wants them.

    Now the hospital-student match looks like it’s the other way – there are more jobs than there are doctors, but the reverse is almost exactly what this proposal is.

    * it’s slightly more complicated, but not much.

Please to post comments

Comments are closed.