Wrong Person May Be Identified 20 Percent of the Time With Facial Recognition Software

Facial recognitionFBIAs of June of this year, enough states had merged their databases of driver's license photos with facial recognition photos to subject 120 million Americans to a permanent police line-up. Some states exercise a little discretion in whom they allow to run searches on those databases, while Ohio permits roughly 30,000 cops, court personnel, and random hangers-on to go trawling for familiar faces. Either way, though, it's a crapshoot for those of us whose mugs have been digitized and stored. As it turns out, facial recognition software may kick back a bad identification about one time in five—at least, that's what official parameters allow.

According to documents the Electronic Privacy Information Center extracted from the FBI with the legal jaws of life known as a FOIA lawsuit:

NGI shall return the correct candidate a minimum of 85% of the time, when it exists in the searched repository, as a result of facial recognition search in support of photo investigation services.

NGI shall return an incorrect candidate a maximum of 20% of the time, as a result of facial recognition search in support of photo investigation services.

By contrast iris searches are supposed to return the correct candidate a minimum of 98 percent of the time, and the incorrect candidate a maximum of 10 percent of the time. Fingerprints are supposed to return a minimum of 99 percent correct candidates and a maximum of two percent incorrect.

Those interesting comparisons come at the end of the very dry Next Generation Identification System Requirements Document (PDF), dated October 1, 2010, in the "Accuracy" section. While technology is always a work in progress, and results may have improved since then, that false-positive rate, as EPIC notes, "is much greater than expected" for a system that's already being deployed and used to match your lovely driver's license photo to a blurry security cam shot of some jerk who knocked over a liquor store.

Editor's Note: We invite comments and request that they be civil and on-topic. We do not moderate or assume any responsibility for comments, which are owned by the readers who post them. Comments do not represent the views of Reason.com or Reason Foundation. We reserve the right to delete any comment for any reason at any time. Report abuses.

  • Rich||

    NGI shall return an incorrect candidate a maximum of 20% of the time

    "We never claimed it was a perfect system!"

  • Sevo||

    Cut us the same slack you cut Apple!

  • Paul.||

    Ok, I hereby refuse to pay a dime for your services and products.

  • Agammamon||

    Using the same excuse as the court's do when not allowing review of the cases of already executed prisoners - closure is the most important thing.

    Somebody was the victim of a crime, we banged up someone as the perpetrator of that crime, what more do you want?

  • Paul.||

    Only 20%?

  • some guy||

    I was thinking the same thing. Facial recognition is a hard image processing task even under ideal conditions where you have complete control of the subject and a small sample set. 120 million is not a small sample set.

  • Homple||

    I add this to my folder labelled "if called to jury duty" along with "eye witness", "trained dog", "forensics lab", "prosecuting attorney itching for higher office"....

    There's enough crap pulled every day to make a reasonable person morbidly skeptical of anything coming from an "authority".

  • AlmightyJB||

    Yeah, I'm on Jury Duty this week. Trial starts tomorrow. I will certainly have my skeptical cap on as always. Gonna be a tough case though.

  • Neoliberal Kochtopus||

    Criminal or civil?

  • AlmightyJB||

    Criminal

  • SIV||

    Malum prohibitum or malum in se?

  • Francisco d Anconia||

    You couldn't bring up jury nullification to get out of it?

  • AlmightyJB||

    That's kind of a catch 22 isn't it? That said, I would not be attempting to nullify the specific charges in this case.

  • AlmightyJB||

    Concerning nullification, I was thinking going into jury duty about what I would do if I pulled a drug case. You're specifically told that you must follow the law and specifically asked if you have a problem with that.

  • Francisco d Anconia||

    No way I'd ever get selected. I wouldn't lie, so the answer would have to be, "It depends on the law."

  • Rich||

    *** rising intonation ***

    Jury nullification is the law.

  • AlmightyJB||

    Yeah, I'm with you. I wouldn't lie, but i would definitely let my opinion be known before they booted me.

  • Kyfho Myoba||

    Hey, judgey-wudgey can tell me anything he wants to - I got no problem with that whatsoever. I just know that when I get in that deliberation room I'll do what I god damn well please. Like the AlmightyJB says, I can find reasonable doubt in anything the gubbmint says. Plus, I'm in Indiana, and the state constitution says specifically that I can judge fact AND law. And if he asks me if I can be impartial, well, of course I can, that's got nothing to do with whether or not the statute conforms to the constitution.

  • SIV||

    Malum in se then?

  • AlmightyJB||

    Yes, most definitely.

  • some guy||

    Just don't send any kids to 50 years in prison for breaking and entering.

  • marshaul||

    Why would you encourage people to use jury nullification to "get out of it"?

    Frankly, the people have every moral prerogative precisely TO lie about their intentions, if that's what's required for them to be "permitted" to exercise what is their right, and quite literally the ONLY check the individual citizen has on government. (Would you give up your guns rather than lie about owning them?)

    At the very least, I can see no reasonable justification for intentionally brining it up precisely to "get out of it".

  • Paul.||

    I've seen you posting here for a while, JB, and while I don't track each individual closely, I presume your Libertarian Bona Fides are at minimum, a strong B+ or greater.

    Having said that, and assuming I'm correct, how in holy hell did you not get thrown out during selection?

  • AlmightyJB||

    I was actually surprised I was selected. I thought I was going to be booted.Given the charges I think they were having trouble finding 12 people anyways.

  • Austrian Anarchy||

    At least they finally got off of that "100% certainty" crap about fingerprints. It took the Spanish to beat it into their heads, but hey, a W is a W.

  • Fist of Etiquette||

    Yeah, like the wrong door never got kicked in before this dodgy app went into use.

  • Paul.||

    Yeah, but now they have a dodgy app to help them.

  • Banjos||

  • The Rt. Hon. Serious Man, Visc||

    That kid's going to grow up to be Lex Luthor. I hope his father gave him so advice on the importance of owning land.

  • playa manhattan||

    /Dwight Shrute

  • AlmightyJB||

    Lindsey Lohan can be his sidekick.

  • Paul.||

    Las Vegas police took the boy into custody when the airplane landed and the incident is under review by the Transportation Security Administration and Delta.

    However, the TSA countered that during that day, they confiscated three toy GI Joe guns, a bullet shaped charm on a bracelet, two belt buckles with western style pistols engraved upon them, an a suspicious batch of ostomy supplies from an octogenarian.

    The problem in this country, is we punish failure.

    /Bill Maher.

  • Ken Shultz||

    "As of June of this year, enough states had merged their databases of driver's license photos with facial recognition photos to subject 120 million Americans to a permanent police line-up."

    Best of all, you don't even need probable cause. Eh, I guess that concern's kinda passe now.

    The Fourth Amendment is so 2012.

  • Paul.||

    Uhm, it's so 1987. Actually, I was being stopped at "Border Patrol" checkpoints in the 70s when I lived in New Mexico and I was no-fucking-where near a border. I'd say the 4th is so 1776.

  • thebitterconsumer||

    Yea, that's bad. 20% is well enough for reasonable doubt, but this software can be used in many other contexts.

    http://thebitterconsumer.wordp.....vics-once/

  • SIV||

    Whenever I see the "Follow Reason on Instagram" I think "KM-W mirror selfies", but no.

  • johnl||

    Does anyone know the quick explanation of what these targets mean? How is it consistent to be right at least 85% of the time and wrong no more than 20% of the time? And not redundant?

  • Rich||

    I'll give you a hint: The answer to all your questions begins with an "F".

    Also, the "correct" requirement contains the phrase "when it exists in the searched repository"; the "incorrect"
    requirement does not.

  • Kyfho Myoba||

    Because FYTW?

  • Ken Shultz||

    "By contrast iris searches are supposed to return the correct candidate a minimum of 98 percent of the time, and the incorrect candidate a maximum of 10 percent of the time."

    I'd use that as your guide for comparison purposes.

    98% is probably how often they find the right guy--when they know he's in the database for testing purposes.

    Sometimes the search probably finds no results. It's not just either they find the right guy or the wrong guy.

    Then they're saying it shouldn't return the wrong guy more than 10% of the time. Sometimes the search probably returns multiple results.

    Anyway, that's my guess. Am I right?

    Then I'll take "The Fourth Amendment" for $600, Alex.

  • Ken Shultz||

    "Sometimes the search probably returns multiple results."

    In other words, sometimes it probably returns both the right guy and some other innocent guy, too, so it counts both towards the 98% returning the right guy and counts towards the 10% returning the wrong guy, too.

    They need to listen to some reggae. That's probably the problem with all those NSA guys. They got no Bob Marley in 'em. They're Air Force people. Probably never burned any commerce--not even once. You do that a couple of times in life with a nice hippie chick? and you don't want to monitor the worlds' phone calls.

    http://www.youtube.com/watch?v=_GZlJGERbvE

  • Archduke von Pantsfan||

    Breaking: Some people have different preferences from others.

  • Rich||

    I do end up flossing with his moustache hairs from time to time but it's all part of the fun.

    Uh, huh.

  • Paul.||

    Anyone remember when Tats were relegated to ex-felons and guys who had served in the Navy?

    Yeah, I remember those days.

    Way to take the mystique out of the tattoo, hipster doofi.

  • Rich||

  • Winston||

    Since the public agrees with Obama then I guess libertarianism is winning!

  • Archduke von Pantsfan||

    Did anyone see tonight's Frontline on the NFL, football, and concussions?

    What will the NFL do after first high schools and then colleges ban tackle football? Won't be long now.

  • Paul.||

    Real Sports with Bryant Gumble/Gumbel (I ain't lookin' it up) covered this a few days ago.

    Apparently, there's an ex football player who has Lou Gerrig's disease or something. Therefore: Head injuries and concussions.

  • Snark Plissken||

    And since the solution to football head injuries is less protection rather than more, it will never be implemented.

  • ||

    Jerry Sandusky's son, who works for the Cleveland Browns, was arrested in Fargo for DUI.

    JOKE OVERLOAD. DOES NOT COMPUTE. ERROR. ERROR.

  • CatoTheElder||

    Way OT: "Kathleen Sebelius: HealthCare.gov simple, user-friendly"

    I'm not creative enough to have made this up!

  • Generic Stranger||

    Also not creative enough to have avoided SFing the link.

  • Archduke von Pantsfan||

    We secretly replaced the NY Rangers with the Bakersfield Condors. Let's see if anyone notices.

  • Francisco d Anconia||

    Didn't they almost get beat by the Mystery Eskimos?

  • Winston||

    Why not the Cleveland Spiders?

  • Hyperion||

    Wha da fook is that thang?

    Something has been sighted that is so fugly that it can only be another Obama appointee. Ye gawds, what the fuck is that thing? Obama, more than anyone in history loves him some fugly butch dykes. He's so sorry that he was only able to bag him a wookie, as he obviously wants to be butch banged by the foogliest bootch dyke on the planet. Fucking gross!

  • Steve from down yonder||

    Are you suggesting ugly people can't contribute to society?

  • Paul.||

    I would suggest that they're just as capable of destroying it as photogenic people are. But I'm a glass-is-half-empty kind of guy.

  • Hyperion||

    Really? Is that the way you interpreted that? The ugliness I am alking about originated from the inside hatefulness of leftwing ideology and seems to soon manifest itself on the outside.

  • Ken Shultz||

    I hope they reject her.

    It's like on The Price is Right. You always turn down the first showcase.

  • C. Anacreon||

    I think she looks like the offspring of Paul Williams and Captain Kangaroo.
    Kids, ask your parents.

  • VG Zaytsev||

    She's a fucking beauty queen next to Obama's consigliore.

  • cavalier973||

    OT: Here's a little something to feed your inner conspiracy theorist (concerns Miriam Carey)

  • cavalier973||

    I love it. I wish I had the time, resources, and expertise to pull this kind of prank.

  • Rimfax||

    Biometric systems use templates to represent faces or retinas or irises or fingerprints. When a sample is collected, it is reduced to a template which is compared to the templates in the datastore, generating a confidence score. For a positive match, the confidence score has to exceed an arbitrary threshold and be the highest confidence score. If the dataset is small enough with distinct faces, no glasses, and no smiles, you can guarantee very high positive matches and very low false positive matches. The numbers provided are meaningless without specifying the size and nature of the dataset. The threshold can be lowered to raise your positive match rate, but at risk of raising your false negative rate, especially if your samples might include subjects not in the dataset.

  • Night Elf Mohawk||

    I wouldn't be surprised if recognition software is still better than eyewitnesses when it comes to strangers. But, even if it's not, it's not like the prosecutors tell the judge there was a potential match and they send you right off to prison.

GET REASON MAGAZINE

Get Reason's print or digital edition before it’s posted online

  • Video Game Nation: How gaming is making America freer – and more fun.
  • Matt Welch: How the left turned against free speech.
  • Nothing Left to Cut? Congress can’t live within their means.
  • And much more.

SUBSCRIBE

advertisement