Popular Mechanics on Faulty Forensic Science
Popular Mechanics has a terrific cover story this month on the crumbling integrity of forensic science. Here's a taste:
The scientific method is instrumental to our understanding of the physical world. To scientists, the process is sacrosanct: Research your topic, generate a hypothesis, test the hypothesis, analyze your data and then publish the results for peer review. Forensic science, however, was not developed by scientists. It was created by cops—often guided by little more than common sense—looking for reliable ways to match patterns from clues with evidence tied to suspects. What research has been done understandably focuses on finding new techniques for putting criminals in jail.
In the academic community the legal sciences get a comparative trickle of federal funding. In 2007, the National Institute of Justice awarded 21 grants for forensic research (excluding DNA) totaling $6.6 million; the National Institutes of Health awarded 37,275 grants totaling $15 billion. And without a wealth of statistically defensible research to back up their evidence, forensic examiners generally rely upon their own intuition and the experience of their colleagues. "You can't take a few case studies and say, 'Oh, it worked on these people; it must be reliable,'" says Karen Kafadar, an Indiana University statistics professor and a member of the NAS committee. "That is hardly a placebo-controlled, double-blind randomized trial."
The article includes a skeptical look at four common forensic specialties, including fingerprint analysis, ballistic evidence, trace evidence, and biological evidence, and explains how none are as certain as they're often portrayed in the courtroom.
For more on this, be sure to check out the paper Roger Koppl wrote for the Reason Foundation on how to introduce real scientific rigor to the forensic process, or the piece on the same topic that Koppl and I co-wrote for Slate.
Editor's Note: As of February 29, 2024, commenting privileges on reason.com posts are limited to Reason Plus subscribers. Past commenters are grandfathered in for a temporary period. Subscribe here to preserve your ability to comment. Your Reason Plus subscription also gives you an ad-free version of reason.com, along with full access to the digital edition and archives of Reason magazine. We request that comments be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of reason.com or Reason Foundation. We reserve the right to delete any comment and ban commenters for any reason at any time. Comments may only be edited within 5 minutes of posting. Report abuses.
Please
to post comments
Hmm.... "Science" being treated with too little skepticism because of govt involvement and media propaganda? I *cough*climate*change*cough* wonder what else we can't trust.
Really the worst thing is the fight to keep lineups unblinded. Modern police departments still don't use 19th century error correction mechanisms.
johnl, would you care to elaborate? Just interested, no axe to grind, unfamiliar with blinded/unblinded line-up methods.
I'm no expert either, but I believe he is talking about the idea that the police should use "blind" questioners in lineups, so that they don't contaminate the witness's response.
Similar in principle to a doctor in a double-blind study not knowing if the pills he is giving out are real or a placebo, the policeman working with the witness should not know which person the police already suspect.
when i first picked up the mag, i immediately checked the byline thinking PM had hired Radley
Thanks, Space.
So before the 19th century, the contaminating nature of subliminal influence was unknown?
Inneresting.
They mention "Daubert" with feeling the need to explain it. Popular Mechanics has turned from a mindless, middling magazine into something for thinking people. I am impressed.
Anarch yes before the 19th century, people did not talk about experimenter bias. Experiments were not blinded in the old days. Fra Mendel's mid 19th experiments were not blinded, as this was not the custom at the time, and this presented him with an occasion of sin that was unfortunately too much for him.
It's rare for USA police lineups to be blinded.
A lot of people (and jurors) aren't told that forensic evidence is usually not an either/or phenomenon, but often requires a judgment call. Such-and-such a soil sample resembles another one, but isn't exact. Fingerprints are usually partial and/or blurred, and cannot be absolutely matched. A blue fiber on clothing might be "consistent with" your blue jeans, but a lot of other clothing, too. Forensic evidence usually doesn't prove a person is guilty, it just increases the Bayesean probability.
Tastes differ.
the biggest problem is TV such as CSI, but it goes beyond that. pop culture has put tons of faith in the abilities of these guys in labsuits and that puts a picture in jurors' heads that the experts are always right.
that's the only explaination i can think of when it comes to cases based off of junk science like bite marks.
I never understood ballistics evidence. Aren't modern firearms essentially identical as they come off the assembly line? Given the tight manufacturing tolerances of today's firearm, if there were any significant statistical variation in the parts the guns couldn't even be assembled. Variances will start to occur because the production tooling starts to wear out. If, for example, the barrel rifling machine starts to wear barrels won't be perfect, but I would expect every gun in a batch to be identical. Barrels, for example, are hacked off pieces of long pipe. Every gun made from the same pipe should have identical rifling.
Bob, I would think they would be near identical when brand new and unfired.
But as a person puts rounds through a gun, they will all wear differently. Mainly due to types/brand of ammunition, as well as the owner's cleaning frequency and technique.
hi would you be intrested in a in blogroll link exchange. let me know....
Forensic evidence usually doesn't prove a person is guilty, it just increases the Bayesean probability.
Do you want to switch to Door Number 2?
Seriously, I wonder whether most jurors have the, um, sophistication to handle probabilitic notions, Bayesian or otherwise.
Bob -
what makes the pipe so angry that it gets hacked off?
in general, people are put in jail to satisfy public demand. our politicians and bureaucrats have sold us on an incarceration society and americans love it. just like the drug war it makes all the nazis look like they are doing something, and therefore deserving of their budgets. meanwhile, people rarely think about what it's like to be locked up or who winds up there.
don't get me wrong, i'm not soft on crime. but if you take into account information such as faulty science, coupled with the techniques cops use to extract info from suspects: the real criminal in the c/j system is the system.
(not to mention the drug war, it can ruin whole countrys full of people who must be evil cause they're not americans. not real ones like us here in the u.s.a.)
as far as fingerprints go, i don't buy this "they're all unique" bs, or the snowflake bs for that matter. i say effing prove it. you can't. you can only postulate and show evidence. it can't be proven.
This implies Hollywood could be wrong about something important. I disagree.
Another beer, Sir?
I read that article a couple of weeks ago (I'm a subscriber). Very nicely done. The money quote, which Radley points out, is "Forensic science, however, was not developed by scientists. It was created by cops--often guided by little more than common sense--looking for reliable ways to match patterns from clues with evidence tied to suspects."
"All you can do is look back and say, 'Was that the best we could do?'"
i'm overwhelmed with contempt for that individual...
Worst offenders: Writers who make exaggerated claims for "criminal profiling." I cringe whenever I hear a TV sleuth talking about the McDonald triad as if he/she were quoting holy writ.
what makes the pipe so angry that it gets hacked off?
I wasn't using slang, I was using "hack" in the sense of "to cut or chop".
But as a person puts rounds through a gun, they will all wear differently. Mainly due to types/brand of ammunition, as well as the owner's cleaning frequency and technique.
Has this actually been tested in a properly controlled study? The number of, say, properly cleaned Glock 17s shooting stock Federal ammo must run in the thousands.
Popular Mechanics lost credibility with me when they first defended FEMA's "truth by simulation" explanation for the collapse of the towers on 9/11, which was later proved by NIST to be inaccurate anyway - via another simulation. Meanwhile, all the evidence gets shipped to China and destroyed - score another one for science, and reason.
They also believe in the Moon landing. Fools! Don't they know that NASA sent actors to the Moon to stage it all?
@ransom
Yeah, hearing that crap from the "expert" that sent me to prison would probably result in my going to prison again.
Only this time it would have to be a different expert testifying against me.