"Policing the Police: The Impact of 'Pattern-Or-Practice' Investigations on Crime"

An interesting draft study by Harvard economics professors Tanaya Devi and Roland G. Fryer Jr.

Here's the abstract:

This paper provides the first empirical examination of the impact of federal and state "Pattern-or-Practice" investigations on crime and policing. For investigations that were not preceded by "viral" incidents of deadly force, investigations, on average, led to a statistically significant reduction in homicides and total crime.

In stark contrast, all investigations that were preceded by "viral" incidents of deadly force have led to a large and statistically significant increase in homicides and total crime. We estimate that these investigations caused almost 900 excess homicides and almost 34,000 excess felonies.

The leading hypothesis for why these investigations increase homicides and total crime is an abrupt change in the quantity of policing activity. In Chicago, the number of police-civilian interactions decreased by almost 90% in the month after the investigation was announced. In Riverside CA, interactions decreased 54%. In St. Louis, self-initiated police activities declined by 46%. Other theories we test such as changes in community trust or the aggressiveness of consent decrees associated with investigations—all contradict the data in important ways.

And the whole study is here. According to my brother Sasha, who teaches at Emory School of Law and who knows a lot about this sort of economic analysis generally, though doesn't specialize in policing:

I can't immediately see anything invalid about this sort of analysis. A few things to keep in mind, based on these guys' credentials: (1) Harvard applied micro econometricians are pretty serious, (2) NBER papers are pretty serious, and (3) their acknowledgments footnote cites a bunch of serious applied micro econometricians. More to the merits, their introductory section (pp. 2–7) seems pretty serious about looking into (and rejecting) alternative explanations for their findings.

In my view, the biggest challenge to this sort of analysis is that whether a police brutality video goes viral isn't exogenous. Maybe such videos are more likely to go viral (1) when the behavior in the video is more egregious, which is more likely in cities with exceptionally bad police-community relations; (2) when the cities have exceptionally bad police-community relations (regardless of whether the behavior in the video is more egregious); etc. If that's the case, then the mere release of the video can result in increased crime, because of rioting, because the community is less likely to cooperate with preventive police activity, because police departments change their behavior to avoid negative PR, etc.—and the investigation doesn't play an important role. The authors reject this explanation because they also look at other cities that had viral incidents but no investigations, but whether a city with a viral incident has an investigation also isn't exogenous. So their identification assumptions aren't airtight; it's not exactly a natural experiment.

That said, it's an interesting result that a bunch of hot econometricians thought was worthwhile.

I would naturally be glad to post links to serious substantive criticisms of the study as well (whether of the finding that some such investigations decrease homicide and other crime, the finding that other investigations increase homicide and other crime, or both findings).