Number-Crunching the Courts
It's time to bring data-driven oversight to the nation's halls of justice.
How many U.S. citizens end up pleading guilty in a courtroom each year without ever getting access to an attorney? In which counties are pre-trial detainees least likely to obtain release through bail? Even know-it-all leaker Edward Snowden may not have the answers to these questions.
As federal operatives eye-grope our metadata and local law enforcement agencies increasingly establish "reasonable suspicion" based on computer algorithms that predict where crimes might happen, our constitutional protections regarding the right to counsel and due process grow even more important.
And yet how effectively can we assess the strength of these protections if we have little or no information on which to base our assessments? If you want to know how the latest Shafer Vineyards cabernet compares within the larger universe of California reds, Wine Spectator is there to give you an authoritative, easy-to-comprehend ranking. If you want to determine which Ivy League school has the best faculty/student ratio, U.S. News & World Report makes that easy.
A similar resource for our court system doesn't exist. "There is no way to compare how counties are performing basic legal services all across America," the journalist and attorney Amy Bach writes in an email. "While [some] organizations attempt to monitor courts, they do so in isolation for intra-court use only."
Bach hopes to change that. Three years ago, after writing Ordinary Injustice, a calmly reported polemic of how expedience, lax adversarialism, and other institutional lapses are systematically eroding the quality of American courts, Bach founded a nonprofit called Measures for Justice. Its mandate: to develop a Justice Index, a standardized set of measures that will make it easier to assess how well courts are performing, both on an individual basis and in relation to their peers.
Many organizations, including the American Bar Association, issue guidelines designed to help courts operate in a fair and accessible manner. And courts do compile substantial information about themselves, some of which they publicize. For example, the National Center for State Courts, an independent, nonprofit court improvement organization, has developed a set of performance measures called CourTools. They include what a given court's cost per case is, what percentage of the people it summons for jury duty end up serving, and similar measurements. But only a handful of states use CourTools to disseminate information about their courts to the public. And currently they do so on an individual basis-there's no one central repository where courts can be evaluated in relation to their peers.
There are 3,143 counties in the U.S., and each one has a state criminal trial court. To start, Measures for Justice is focusing on such courts, where approximately 21 million cases are considered each year. "Unless you've committed a federal crime or your case has been appealed, this is where justice happens for most people in the U.S," notes Bach.
One reason a resource like the Justice Index doesn't yet exist is because our court system is so decentralized. Laws and procedures vary from state to state, courts are given a fair degree of latitude in establishing how they conduct business on a daily basis, and there are no common standards regarding the sorts of information they are mandated to collect and report. So over the last three years, Bach and her colleagues have been drafting and testing potential "illustrative measures"—a set of indicators, based on publicly available data, that aim to communicate in a data-driven snapshot a given court's fairness, accuracy, cost efficiency, and ability to improve public safety.
In a white paper that Measures for Justice plans to publish soon, it describes eight of these performance measures more fully and details some of what it found when using them across different courts. (*) For example, if you're planning to commit a crime but you're on a budget, go to Northern California's Contra Costa County, which has the most affordable median bail amount out of the 39 counties Measures for Justice assessed-under $3,000. The median bail amount in Los Angeles, in contrast, is closer to $50,000. The major reason for the difference: Los Angeles County uses a standard bail schedule, rather than judicial discretion, to impose bail, and automatically assigns a bail of at least $20,000 to almost every felony.
In the realm of pretrial detention for felony defendants, it's Honolulu that's the outlier, duration-wise. While more than a dozen counties in the survey released such detainees within an average of 10 days from their arrest, the average pre-trial stay in Honolulu was closer to 50 days, almost twice as long as any other county in the survey. That, the report concludes, suggests "considerable inefficiencies in processing the release."
How many similar patterns are our courts hiding? And who will fully investigate their import? Measures for Justice has a small staff and a modest budget. At the moment, it gets its funding from a handful of private foundations and the U.S. Bureau of Justice Assistance, a federal agency that helps develop criminal justice policies for local and state administrations. Measures for Justice is currently refining its methods of data collection and assessment via a pilot program located in Milwaukee County, Wisconsin, and over the next four years it hopes to expand its coverage to hundreds of counties in over 15 states. (**) In other words, it would have to scale considerably to become a comprehensive, one-stop watchdog overseeing our court system.
What Measures for Justice illuminates already is the scattershot way newspapers have traditionally taken up that monitoring role. While pulp-and-ink loyalists lament the darkness that is destined to descend upon America when the last old-fashioned courthouse reporter takes his early retirement buyout, the truth is that most newspapers have historically devoted far more staff, editorial space, and even wonkish number-crunching to covering what goes on inside sports stadiums than courthouses. Was there ever any daily in the land, even when newspapers were at their fattest and most profitable, that published, day in and day out, for months on end, the prosecutorial batting averages of the local district attorneys?
Even today, as data-driven journalism becomes more commonplace, most newspapers still spend far more time presenting courtroom malfeasance through the lenses of novelty, drama, and shock value-the rogue judge who bullies defendants into waiving their rights, the poor wretch who spends a year in jail after being arrested on loitering charges because he slips through administrative cracks-than they do engaging in the sort of mundane pattern analysis that can illuminate system-wide problems and form the basis for potential reform.
On its website, Measures for Justice positions itself in part as a tool "to defend against austerity measures being implemented nationwide." Courts that are given high rankings on the Justice Index can use it as a justification for maintaining their budgets, the organization advises. And even courts that score low can use their poor performance to "lobby for an increased budget."
That's an understandable pitch, as the data that Measures for Justice seeks would be nearly impossible to collect without significant cooperation from the courts themselves. But budget increases are only one potential outcome. If the Justice Index reveals that County A is prosecuting murder charges with a 20 percent higher success rate than County B, while only spending 60 percent as much per case, the taxpayers who live in County B are going to want to know why. The Justice Index should spur competition and innovation, and thus potentially lower court costs. Courts whose policies and procedures are generating positive outcomes will emerge as models for others to emulate. Courts that are performing poorly will face pressure to adapt.
If citizens and private organizations embrace Measures for Justice, and help it grow into an aggressive, independent watchdog, it will also serve as a reminder to our government that intensive surveillance is now a two-way street. Instead of simply relying on whistleblowers or settling for sporadic narrative-driven oversight, it's now possible to monitor and assess government behavior using the same sort of sweeping and perpetual omniscience with which it monitors us. Ideally, we'll eventually bring this attention to all facets of state power. The court system, where it routinely exerts its greatest influence over the citizenry, is an apt place to start.
* This article originally stated that Measures for Justice has been drafting and testing a set of eight preliminary illustrative measures. Measures for Justice founder Amy Bach clarifies that the organization has actually tested more than eight measures to date, and she notes that the organization may ultimately include more than that in its final Index. In its white paper, Measures for Justice presents eight of the measures it has tested so far.
** This article originally stated that Measures for Justice hopes to expand its coverage to 50 counties over the next four years. That statement was based on outdated information taken from the organization's website.