Forthcoming NAS Study May Shake Up Forensics System

A forthcoming study from the National Academy of Sciences is expected to send shockwaves through the criminal justice system.

People who have seen it say it is a sweeping critique of many forensic methods that the police and prosecutors rely on, including fingerprinting, firearms identification and analysis of bite marks, blood spatter, hair and handwriting. The report says such analyses are often handled by poorly trained technicians who then exaggerate the accuracy of their methods in court…

Legal experts expect that the report will give ammunition to defense lawyers seeking to discredit forensic procedures and expert witnesses in court. Lawyers could also use the findings in their attempts to overturn convictions based on spurious evidence. Judges are likely to use the findings to raise the bar for admissibility of certain types of forensic evidence and to rein in exaggerated expert testimony.

Law enforcement organizations have tried to derail the report nearly every step of the way. The report's critique of forensic evidence is much needed, but the proposed solution doesn't sound promising:

It concludes that Congress should create a federal agency to guarantee the independence of the field, which has been dominated by law enforcement agencies, say forensic professionals, scholars and scientists who have seen review copies of the study.

I wouldn't mind seeing an agency within the Department of Justice devoted to investigating and prosecuting cases of forensic fraud. If prosecutors are conspiring with or pressuring experts to deny criminal defendants a fair trial, that would be a due process violation and under the Fourteenth Amendment, the federal government would be permitted, or even obligated, to step in. Certainly Mississippi, for example, has neglected its duty to ensure that its citizens accused of violent crimes are given a fair trial.

But setting aside wilful and criminal misconduct by forensic experts, the problems with the forensics system aren't going to be resolved by creating a new federal bureaucracy. Lack of federal oversight isn't the problem. According to the New York Times article linked above, for example, the NAS report is particularly critical of the FBI crime lab, long considered the gold standard in forensics, and whose technicians often advised state crime labs on best practices.

The problem with criminal forensics is the government monopoly on courtroom science in criminal trials. In too many states, forensic evidence is sent only to state-owned or state-operated crime labs. There's no competition, no peer review, and in some cases, crime lab workers either report to or can be pressured by prosecutors when test results don't confirm preexisting theories about how a crime may have occurred. This sort of bias can creep in unintentionally, or it can be more overt.  But studies show it's always there. The only way to diminish is to bring competitors into the game, other labs who gain by revealing another lab's mistakes.

Every other area of science is steered by the peer review process. It's really unconscionable that criminal forensics—where there's so much at stake—has existed and evolved so long without it. The fact that so many people have been convicted solely based on pseudo-sciences like bite mark and hair and fiber analysis, for example, ought to scare the hell out of us.

Roger Koppl wrote an excellent paper for the Reason Foundation (pdf) that outlines the problems with criminal forensics, and offers a series of reforms for ensuring that the science jurors hear in the courtroom is actually science. Koppl and I also co-wrote a shorter piece in Slate making essentially the same points. Koppl's proposals sound radical, but only because we're so used to the current system. What he's proposing is really little more than applying basic scientific principles like peer review, blind testing, and repetition to the evidence and opinions currently presented in criminal cases as actual science.

And we need to start treating criminal forensic science like actual science, with all the skepticism and repetitive testing that comes with it. What we don't need is another layer of government bureaucracy that imposes a series of negotiated, compromised-for standards and practices, then fails to properly enforce them.

Comments

[Paul]

I believe that all things like should be reviewed in pursuit of justice. I fear, however, that clever defense attorneys will use this to discredit good and reasonable forensic analysis.

Sort of like that study that showed adolescents critical thinking wasn't fully developed. I remember a defense attorney on NPR was shouting that "science had proved" that anyone under 25 wasn't responsible for their crimes.

[HammeredHead]

Keep up the good work Radley. I never would have heard of the travesty of justice occuring in Mississippi without your reporting. I wish your columns could be more widely reported.

[JSteele]

Its refreshing to see this here and read the article. I have been a chemist floating around the edges of the legal system as a defense expert for about a decade. I have done a fair number of cases and the quality of the analytical chemistry in those cases has ranged from laughable to very good. One of the worst was the FDA forensic lab which chose never to follow FDAs own Good Laboratory Practice regulations. So far as to not even have written, signed and dated method protocols or any formal validation studies for their methods. Many abuses which in my real world job in the pharma industry would be cause for criminal prosecution of our company. The Cook county drug identification unit was also rife with mistakes and quality control errors which not even a student laboratory could tolerate. Frankly too many such examples are out there and the situation has long needed true reform. Donald Kennedy fought for some time just to get this report out the door. Its what could be done right now late as the work might be. A lot of people have chewed away at the edges of this problem for the last 30 years and more and progress has been extremely slow. The greatest single reform would come from just upgrading the quality control and validation procedures up to the well worn standards for any well run quality assurance lab in the Pharma arena. Simple competence and reasonable oversight within a well administered quality assurance environment would go a long long way to correcting the abuses.

[NAS]

Hip hop is dead.

[Dave W.]

What we need to do is to give defense money to criminal defendants. Lots more than we do now.

Bad science is one problem, but it is mere ly part of a larger problem.

[Tonio]

I believe that all things like should be reviewed in pursuit of justice. I fear, however, that clever defense attorneys will use this to discredit good and reasonable forensic analysis.

Yes, which is unfortunate but unavoidable. But it is better that ten guilty men go free than one innocent man be imprisoned.

The real blame lies with the people who turned the forensics labs into little more than rubber-stamps for the police and prosecutors.

"Qui custodiet ipses custodiens?" and all that.

[R C Dean]

What we need to do is to give defense money to criminal defendants. Lots more than we do now.

I'm still toying with the idea of merging the public defender's office with the district attorney's office, and assigning attorneys randomly to the prosecution and the defense.

The only hangup I can see is maintaining a confidentiality firewall between the prosecuting attorney on a given case and the defending attorney on that case, when they are both in the same office.

[Bramblyspam]

You don't necessarily have to have competition in order to increase the reliability of the system. If you can set up a system whereby every sample is tested double blind, that should do the trick.

[Tyler]

But it works on CSI!!!

[Mad Max]

In the federal system (and states with evidence rules modeled on it) there's a tool which judges can use if they want to which could address some of these problems, if more widely used.

Rule 706 of the Federal Rules of Evidence allows the judge to appoint his own experts in both civil and criminal cases. The judge doesn't have to, and usually he doesn't - usually the only experts are appointed by the parties. Perhaps judges are worried about the expense of experts, or perhaps judges see court-apointed experts as contradicting the adversary system.

The Advisory Committee comments to Rule 706 declare: 'While experience indicates that actual appointment [of court experts] is a relatively infrequent occurrence, the assumption may be made that the availability of the procedure in itself decreases the need for resorting to it. The ever-present possibility that the judge may appoint an expert in a given case must inevitably exert a sobering effect on the expert witness of a party and upon the person utilizing his services.'

Perhaps the forthcoming report will provide a basis for challenging the 'assumption' that the threat of the court appointing its own expert will induce the parties to recruit high-quality experts.

At least in criminal cases, judges should be prompted to make a greater use of their expert-appointing power.

[Mad Max]

The expense of court-appointed experts could be paid out of the savings made by abolishing unconstitutional, wasteful and oppressive federal programs.