Face Masks Confuse Facial Recognition Technology

That's a good thing.

Americans can't agree on whether face masks are a good way to reduce the threat of transmitting COVID-19. We've even turned mask-donning into a symbol of partisan affiliation; those who would make them compulsory everywhere face off against those who refuse them under all circumstances. But we should at least be able to agree that face coverings are a great way to defeat the surveillance state—especially now that the U.S. government has conceded that masks confuse the hell out of facial recognition technology.

"Using unmasked images, the most accurate algorithms fail to authenticate a person about 0.3% of the time," the U.S. National Institute of Standards and Technology (NIST), a federal agency, reported last week. "Masked images raised even these top algorithms' failure rate to about 5%, while many otherwise competent algorithms failed between 20% to 50% of the time."

Notably, the NIST test focused on one-to-one matching of a face against a single photo, as you might do to unlock a cellphone or at a passport checkpoint. One-to-one systems are carried out under conditions of near-ideal lighting and camera placement, and so are more reliable than one-to-many matches of faces against databases that are conducted during surveillance of public places. Masks should be expected to be even more effective at increasing failure rates of one-to-many facial recognition systems.

"The more of the nose a mask covers, the lower the algorithm's accuracy," the NIST report adds of the digitally simulated coverings used in the study. "The study explored three levels of nose coverage—low, medium and high—finding that accuracy degrades with greater nose coverage."

Perhaps more surprisingly, black masks turned out to defeat matching algorithms more thoroughly than did light-blue masks. The researchers speculate that very dark and very light masks might confuse cameras' automatic light-exposure controls. So, the ninja look isn't just aesthetically pleasing, it's also practical from a privacy perspective.

The NIST report confirms fears voiced in a May 22 Department of Homeland Security (DHS) notice that people could take advantage of mask-wearing to defeat surveillance efforts. Part of the recent BlueLeaks hack of law enforcement documents published by Distributed Denial of Secrets, the notice warned:

We assess the widespread use of masks for public safety could likely continue to impact the effectiveness of face recognition systems even after federal or state mandates for their use are withdrawn as portions of the general population will likely continue to voluntarily wear face coverings in public even after restrictions on social gatherings are lifted or until an effective COVID-19 vaccine is publicly available.

The notice also talked of other means of defeating facial recognition technology, including blue and green lasers to blind cameras, "clothing or accessories with images of faces, license plates, or pixelated images" to confuse algorithms, and specialized hats and other accessories that emit infrared light that can wash-out camera images.

I'll add here that I performed a casual experiment with an infrared flashlight clipped to the visor of a baseball cap, and my face was mostly obscured by a white blob on the image transmitted by a home-security camera.

Some clever entrepreneurs have deliberately targeted the market for surveillance-defeating fashion, producing dazzling eyeglasses and distracting clothing intended to drive facial recognition technology to distraction. The tactic is apparently effective, although perhaps at the price of making people look a bit like Elton John impersonators.

Now you can add to the list of effective anti-surveillance tools the fabric masks that some jurisdictions require us to wear in the name of  public health.

That government officials are going to end up awfully conflicted over whether to mandate or forbid masks in the years to come is obvious from the case of Hong Kong. Last October, the Hong Kong government banned the wearing of facial coverings in public places because pro-democracy demonstrators had adopted them to deter surveillance efforts by Chinese authorities. Now, the same officials require the wearing of masks in public places as part of efforts against the pandemic.

It's hard to be a control freak.

Similar concerns prevail in the U.S. as federal and state law enforcement agencies convert databases from sources including the U.S. State Department and state motor vehicle departments into a treasure trove of images against which to match surveillance of public places. As of last summer, the FBI had compiled a collection of 640 million faces to peruse.

Government officials may promote mask-wearing now but, as the DHS notice demonstrates, they're already worried about the effect that normalizing facial coverings has on high-tech surveillance programs. It could be frustrating to spend years developing sophisticated algorithms, and networks of cameras connected to vast databases, only to see the expensive effort thwarted by the popularization of the bandit look.

That's not to say that face masks are necessarily an absolute check on surveillance. As the NIST report points out, concealing the face to one extent or another reduces the reliability of facial recognition technology, but it doesn't completely eliminate matches.

Some security companies—particularly those serving China's police state—claim that their facial recognition technology can work around masks by matching images of people's eyes. "But the system struggles to identify people with both a mask and sunglasses," Reuters reports.

NIST plans to assess such mask-accommodating technology in the near future.

"We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks," says Mei Ngan, an author of the report. "Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind."

NIST also plans to test one-to-many searches to assess the impact of face masks on surveillance of public places.

For privacy-minded people who are skeptical of public health arguments in favor of face masks, the pandemic may prove to be less of a reason to wear face coverings than an excuse to do just that. And we may as well throw in sunglasses and a hat, just to be sure.