Glasses Equipped with Facial Recognition Are in Our Future

Two Harvard undergrads give us a glimpse of the surveillance future.

Most discussions of facial recognition technology contemplate a world in which people walk the streets and drive the roads under the watchful eyes of government surveillance cameras. Those cameras will tag and track those who fail to hide their features, recording their movements for future reference.

But what if one day facial recognition tech becomes so cheap and portable that it can be built into wearable devices? You wouldn't know if somebody calling your name at a bar was an old friend or a con artist working a scam after linking your face to searchable online personal information. That day is now, courtesy of two Harvard University undergrads.

These Glasses Can See Into Your Life

Creators AnhPhu Nguyen and Caine Ardayfio, two Harvard students with undoubtedly impressive careers ahead of them, introduced what they call I-XRAY with a video on Instagram (also on X) of the two taking turns walking around Harvard's campus and the streets of Cambridge, Massachusetts. They shared information drawn from the internet about people they meet, based on identification by facial features. In two cases, the subjects aren't aware of the experiment and the inventors engage them in casual conversation as if they were past acquaintances.

"To use it, you just put the glasses on, then as you walk by people the glasses will detect when somebody's face is in frame," Ardayfio says in a voice-over. "This photo is used to analyze them, and after a few seconds their personal information pops up on your phone."

"The information our tool collects from just a photo of your face is staggering," he adds.

The tool is partially a pairing of hacked Ray-Ban Meta smart glasses and the PimEyes face search engine among what the duo describe as "five established technologies." The glasses have built-in cameras, speakers, microphones, and AI capability, and link with phones via Bluetooth. They also give you a classic Buddy Holly look, if you're style-conscious. The glasses can live-stream directly to Instagram and Facebook, which is important for facial-recognition use.

"We stream the video from the glasses straight to Instagram and have a computer program monitor the stream," details Nguyen. "We use AI to detect when we're looking at someone's face, then we scour the internet to find more pictures of that person. Finally, we use data sources like online articles and voter registration databases to figure out their name, phone number, home address, and relatives' names and it's all fed back to an app we wrote on our phone."

Combining Creativity With Off-the-Shelf Technology

What's amazing is that Ardayfio and Nguyen basically contributed code to connect off-the-shelf consumer technology. Their work didn't require cutting-edge hardware or a controversial law-enforcement-only facial recognition service. They used a $300 pair of smart Ray-Bans and a search engine that costs 30 bucks a month for a basic subscription. Ray-Ban advertises the glasses as "the latest in wearable tech with authentic Ray-Ban design, to keep you connected wherever you go." The New York Times reviewed PimEyes as "alarmingly accurate" and capable of matching people with photos from across the internet. Brought together by young innovators, they prove a powerful combination.

Not that the companies in question are entirely thrilled about the free advertising.

"They have not only demonstrated their point but also unintentionally provided a blueprint for malicious individuals on how to weaponize readily available tools," Giorgi Gobronidze, director of PimEyes, cautioned Biometric Update.

Well, malicious individuals were probably going to figure this one out, anyway. They may have already done so, without bothering to publish the results. You don't head off technological threats by concealing them; you have to accept their existence and respond appropriately.

PimEyes is working on preventing its service from being used this way again. But it's obvious the cat is out of the bag. If two undergrads can craft wearable facial-recognition technology using consumer products and services, what are high-tech researchers with contracts from government security agencies up to? And what will it mean for privacy to live in a world in which accessories worn by cops, spies, terrorists, and anybody with an interest might be the eyes and ears of a surveillance network?

The Electronic Frontier Foundation maintains an Atlas of Surveillance that lets users search for surveillance technology in use by law enforcement agencies in specific communities or along travel routes. It's a fascinating and helpful tool, but one that will be a challenge to keep current as prying eyes and ears become cheaper, more portable, and ubiquitous. The same can be said of the organization's Surveillance Self-Defense Guide, which doesn't yet contemplate a world in which every chance encounter on the street, or each glimpse by a stranger, might record our whereabouts.

Battling Surveillance When It's Everywhere

"As cities become ever more packed with cameras that always see, public anonymity could disappear," John Seabrook wrote in 2020 for The New Yorker. "Can stealth streetwear evade electronic eyes?"

For the piece, he interviewed privacy activists who create clothing that confuses surveillance cameras or even feeds garbage data into their systems to baffle their algorithms.

Interest in anti-surveillance technology has continued since then. Last year, in Vice, Jason Koebler described a high-tech hoodie that actively blinds cameras with built-in infrared LEDs. The garment's developer had previously created a cap designed to confuse surveillance with bogus faces.

In a world of surveillance, we may be entering an arms race between wearable surveillance tools and opposing privacy-protecting fashion.

But most people aren't so surveillance-averse that they'll redesign their wardrobes around shielding their identities and their movements. The most concerned among us might take to wearing infrared-strobing caps and cloaks to defeat the all-seeing eyes of those around us. But it's difficult to imagine the majority of the population going to that level of trouble.

As for legal protections against building surveillance tech into every device imaginable, it's easy enough to foresee it getting passed in some form, and then enforced against clever undergraduates and freelance hackers. But assurances by the powers that be that they're acting with restraint will always be awaiting the next Edward Snowden to tell us just how hollow they are.

Nguyen and Ardayfio may have introduced us to a world in which we'll all soon be living.