How to Bring Real Science Into the Courtroom

A disturbing new report says our criminal courts have been relying on bad evidence.

Radley Balko | February 20, 2009

A forthcoming study from the National Academy of Sciences on the poor quality of forensic science in America’s courtrooms is expected to send shockwaves through the criminal justice system. According to The New York Times:

People who have seen it say it is a sweeping critique of many forensic methods that the police and prosecutors rely on, including fingerprinting, firearms identification and analysis of bite marks, blood spatter, hair and handwriting. The report says such analyses are often handled by poorly trained technicians who then exaggerate the accuracy of their methods in court.

Law enforcement organizations have tried to derail the report nearly every step of the way, and with good reason. Police and prosecutors have been relying on bad science to get convictions for decades. It’s only recently, as the onset of DNA testing has begun uncovering a disturbing spate of wrongful convictions, that some of the criminal justice system’s cottage industry pseudo-sciences like "bite mark analysis" have been exposed for the quackery they are.

The power of DNA to exonerate the condemned has us quickly learning that our courts have for years been corrupted by charlatans and snake-oil salesmen, such as Mississippi’s dubious “bite mark expert” Dr. Michael West and impossibly industrious medical examiner Dr. Steven Hayne; Oklahoma City’s Dr. Joyce Gilchrist; or Maryland’s Joseph Kopera, to name just a few.

The report’s critique of forensic evidence is much needed, but the proposed solution doesn’t sound promising. According to The New York Times, the report "concludes that Congress should create a federal agency to guarantee the independence of the field, which has been dominated by law enforcement agencies."

The problems with the forensics system aren’t going to be resolved by creating a new federal bureaucracy. Lack of federal oversight isn’t the problem. According to the Times article, the NAS report is particularly critical of the FBI crime lab, long considered the gold standard in forensics, and whose technicians often advise state crime labs on best practices.

The problem with criminal forensics is the government monopoly on courtroom science in criminal trials. In too many states, forensic evidence is sent only to state-owned or state-operated crime labs. There’s no competition, no peer review, and in some cases, crime lab workers either report to or can be pressured by prosecutors when test results don’t confirm preexisting theories about how a crime may have occurred. This sort of bias can creep in unintentionally, or it can be more overt. But studies show it’s always there. The only way to compensate for it is to bring competitors into the game, other labs who gain by revealing another lab’s mistakes. Every other area of science is steered by the peer review process. It’s really unconscionable that criminal forensics—where there’s so much at stake—has existed and evolved so long without it.

It wouldn’t be a bad idea to set up some sort of task force within the Department of Justice devoted to investigating and prosecuting cases of outright forensic fraud. If prosecutors are conspiring with or pressuring experts to deny criminal defendants a fair trial, that would be a due process violation and under the Fourteenth Amendment, the federal government would be permitted, or even obligated, to step in. Certainly a state like Mississippi, for example, has neglected its duty to ensure that its citizens accused of violent crimes are given a fair trial.

But if we’re really serious about making a true science out of forensics, we need to fundamentally alter the way forensic evidence is generated for use in the courtroom. Roger Koppl, an economist and forensic expert at Fairleigh-Dickinson University has come up with some excellent suggestions (disclosure: Koppl outlined these suggestions in a report (pdf) for the Reason Foundation, which publishes Reason. Koppl and I have also co-written two articles on this issue). Among them:

• Defendants should be given access to their own forensic experts. For every prosecution expert, defendants should be issued a voucher to hire their own expert.

• Forensic evidence (autopsies, fingerprints, blood samples, and so on) should at least periodically be sent to more than one lab for testing. Even sending just every third or fourth sample to an independent lab would go a long way toward keeping state labs honest (state labs wouldn’t know when other labs would be doing the same testing).

• Forensic experts should refrain from talking with police and prosecutors before conducting their tests. Studies show that exposure to theories about how a crime may have been committed beforehand can bias an expert’s results, even unintentionally. States should hire evidence handlers to shepherd evidence between law enforcement and crime labs without conveying any contextual information about where or how the evidence was obtained.

• State forensic experts should not serve in the same state bureaucracy as police or prosecutors. Ideally, they should report to criminal court judges. Barring that, they should be independent, and not in any way be considered part of the prosecution’s “team.”

• States should conduct periodic statistical reviews of crime lab results, to see if any labs or individual lab technicians are producing statistically unlikely results.

These ideas sound radical, but in truth they amount to little more than applying basic scientific principles like peer review, blind testing, and repetition to the evidence and opinions currently presented in criminal cases as science, but isn’t subjected to the sort of scrutiny and review other sciences are. Forensic science is in bad need of reform, but it needs to be the right kind of reform. What we don’t need is another layer of government bureaucracy that imposes a series of negotiated, compromised-for standards and practices, then fails to properly enforce them

Radley Balko is a senior editor at Reason magazine. This article originally appeared at FoxNews.com.

Comments

|2.20.09 @ 3:06PM|#

Yes, it's true. For eg: http://fieldsobrietytest.info

|2.20.09 @ 3:14PM|#

Great ideas, Radley. Too bad they'll never be implemented.

EscapedWestOfTheBigMuddy|2.20.09 @ 3:23PM|#

Forensic experts should refrain from talking with police and prosecutors before conducting their tests. Studies show that exposure to theories about how a crime may have been committed beforehand can bias an expert's results, even unintentionally.

There is no end to how important this is.

These is an increasingly standard procedure of "blinding" [1] data in particle and nuclear physics, because we have good evidence that knowing the "right" or "theoretical" value causes even well meaning people to bias their measurements [2].

If this can happen in physics (with nothing but careers and bragging rights on the line) it can damn well happen in an emotionally charged field like forensics.




[1] Blinding can take the form of munging the data with a additive or multiplicative constant that will survive the analysis in a way that allows it to be removed at the end, or it can take the form of shielding the analysis team from some details of the experiment inputs, or...

[2] A favorite example here is the evolving understanding of the lifetime of a free neutron.

SpongePaul|2.20.09 @ 3:42PM|#

You know we are in troubla as a nation when the common sense ideas put forth in the article are by the authors own admission may be radical

|2.20.09 @ 3:55PM|#

I'm certain that cops and prosecutors hate these excellent proposals. It makes their job harder and them accountable. Red state, blue state, north or south, it won't matter, the establishment will fight tooth and nail, lie and obfuscate to maintain the status quo. For far too many, it's not about truth or justice, it's about winning the game.

to them I say, "When lives are at stake, it ain't a fuckin' game".

New World Dan|2.20.09 @ 4:09PM|#

Great ideas, Radley. Too bad they'll never be implemented

My wife is a forensic scientist. She is subject to peer review and blind testing. Everyone in her lab has a minimum of a master's degree in forensic science. Of course, she also works in the DNA section of the lab. I can't tell you how well some of the other sections are run. As far as I know, her lab considders bite mark analysis to be largely quackery.

It would be, however, a good thing for defendants to have equal access to the lab. Otherwise, my wife's lab section already implements most of the best practices Balko outlines.

|2.20.09 @ 6:31PM|#

This. Is. Huge.

My modest proposal -- have the courts supervise the labs. At the federal level they do a good job firewalling public defenders from the judges.

Thanks, Radley, for everything you've done to bring this issue into the open. Hope that this will cause more interest in your reporting in particular, and libertarian issues in general.

NPR covered this Thursday night during drivetime. That the issue is finally gaining some traction with liberals is good, as the conservatives haven't been helpful in corrrecting this.

Steve: buzzkill.

SpongePaul: Well said.

JaySub: word.

Douglas Gray|2.20.09 @ 10:10PM|#

Why not have private labs that defense attorneys can go to?

This is on a different subject, but makes the point.

There is a guy back East marketing a device he claims improves gas mileage and gives less pollutants. Recently, the FTC took him to court, claiming his claims were fraudulent.

The FTC hired a scientist who was an expert in cryogenics. He said it was scientifically impossible for the device to work as claimed. But during cross examination, he confessed that he had never studied the internal combustion engine, and didn't really know that much about how they worked!!

The defense hired a scientist who was a recogized expert in internal combustion engines. He said that,according to his knowledge and observations, the device does work as claimed.

The judge, after listening to both experts, and other testimony as well, ruled against the FTC and in favor of the plaintiff. He remarked that the FTC scientist didn't seem to know what he was talking about.

This is the sort of thing we need in forensics, where you have the freedom to call in outside experts.

|2.21.09 @ 8:05AM|#

Why not have private labs that defense attorneys can go to?

There are private labs that defense attorneys can go to. However, they are expensive, as in fact the government ones are.

One of the problems facing defendants is that the State has virtually unlimited resources to throw at a case. While it's true that some cases are not pursued as vigorously once one is determined to be important enough the State pours everything it has into it.

Even relatively well healed defendants simply find they run out of money for attorney's fees and expenses for outside experts long before the State does. With luck, at that point they may get a plea deal that isn't too harsh.

jkp|2.21.09 @ 8:30AM|#

The executive and legislative branches rely on bad science all the time to make policy - why should this surprise us?

VM|2.21.09 @ 10:43AM|#

Thanks Radley! Excellent work as usual!!

|2.21.09 @ 10:14PM|#

Yeah the government is going to spend money to call its precious convictions into question. Even if they were willing to do that out of the principle of justice, can you imagine the vehement outrage by the "victims" and "victims' rights" groups? These people (typically composed of Christians, which is the most ironic fact in modern politics), display bloodthursty revenge against the people whom a jury was convinced did them harm. They whine and scream about "closure" and the notion that those precious convictions that they claim let them feel so safe and sleep so well at night would be called into doubt would be intolerable.

In america, bogus convictions due to junk science are better than letting a guilty person go free, because in America it's now preferable that an innocent person suffer a wrongful conviction and incarceration than for a guilty person to be wrongly acquitted and set free to roam the streets. I find it amazing, and I'm talking about a 1 to 1 ratio here. In the past, it was common understanding that it's better for 1000 guilty people to go free than for 1 innocent person to be wrongly convicted. Nowadays, all that means is 1000 criminals out on the streets hurting children.

The children, the children. Think how many social and political problems we could solve if we banned the use of the world "child" or "children" in all political discourse. Everyone should start screaming and booing the second a politican uses the "C-word" and any law with the C-word in the title should be automatically, per se unconstitutional. The Children's Playground Protection Act... unconstitutional. The No Child Left Behind Act - unconstitutional. The Children's Online Protection Act - unconstitutional. The Children's Tax Increase Act - unconstitional, and so forth.

|2.21.09 @ 10:27PM|#

Think how many social and political problems we could solve if we banned the use of the world "child" or "children" in all political discourse.

But won't somebody [nevertheless] think of the children?!

PS Sex Cauldron? I thought they closed that place down!

|2.22.09 @ 2:27AM|#

Marc: I don't mind people "thinking" about children. My problem is people talking about caring about children. They can think about whatever and whomever they want. Just keep those thoughts to themselves... and don't verbalize them.

Godfrey|2.22.09 @ 4:54AM|#

One thing I've always loved about Radley: he's a fuckin' bulldog. Once he sinks his teeth into something, be it the militarization of our police forces or the above subject, he doesn't let go.

Keep it up, Brotha B.

Godfrey|2.22.09 @ 5:01AM|#

BruceM: what is even remotely ironic about Christians displaying bloodthirsty revenge of any sort?

Time for a quick reread of the Old Testament... or whatever history book happens to be near at hand.

|2.22.09 @ 10:09AM|#

It shouldn't be terribly surprising for the courts to get it wrong when it comes to science; they are simply being consistent. Just look at the pitiful job they do with the Constitution. It's make it up as you go. Why should science be any different?

If they press on in delusion that CO2 is a "dangerous" gas that should be regulated, we are in for it.

I just wish that in all of their quixotic meanderings they would stumble into the U. S. Constitution. Wouldn't that be a novel idea?

ev|2.22.09 @ 6:00PM|#

Sex cauldron is probably one of my favorite Simpsons quotes.

Justen|2.23.09 @ 3:25PM|#

BruceM: what is even remotely ironic about Christians displaying bloodthirsty revenge of any sort?

Time for a quick reread of the Old Testament... or whatever history book happens to be near at hand.

Forget the Old Testament, look at what the actual Christians (not their theological forefathers) got up to. They've been involved in violence and terrorism since the dawn of their faith. The crusades, the inquisition, recurring witch hunts and genocides throughout history. Only the modern evangelical & charismatic movements have really tried to put a happy face on the dark side of that religion.

skepticscott|2.23.09 @ 10:33PM|#

Sorry, but Roger Koppl's suggestions are anything but excellent, and he is an economist, not a "forensic expert". While the notion of lots of competing private laboratories is inherently appealing to libertarians and free-marketeers, a moment's thought would reveal that this would corrupt science and bias results far more than the current system, in which analysts are mainly civil servants, who are not paid by the person submitting the evidence and whose livelihood and job status do not depend on achieving results that will please the prosecutor or police department.