One afternoon in early 2017, at Facebook’s headquarters in Menlo Park, Calif., an engineer named Tommer Leyvand sat in a conference room with a smartphone perched on the brim of his baseball cap. The smartphone, attached with rubber bands to keep it in place, had the camera facing out. This unconventional hat-phone, a rather unfashionable glimpse into the future, possessed a secretive tool known only to a select group of employees. What it could accomplish was truly remarkable.
Inside the room, a small group of men were ecstatically laughing and talking over each other, as captured in a video taken that day. However, the commotion came to a halt when one of them requested silence. The room grew quiet as the demonstration began.
Leyvand turned towards a man seated across the table from him. The smartphone’s circular, black camera lens stared unblinkingly above Leyvand’s forehead like a Cyclops eye, observing the face before it. Two seconds later, a robotic female voice announced, “Zach Howard.”
“That’s me,” Howard, a mechanical engineer, confirmed.
An employee who witnessed the tech demonstration initially assumed it was intended as a joke. However, as the phone accurately started calling out names, a sense of unease washed over him, akin to a scene from a dystopian film.
While the hat-phone, capable of identifying individuals, would be a boon for those with vision impairments or face blindness, it was also risky. Facebook’s previous use of facial recognition technology, specifically for tagging friends in photos, had sparked outrage from privacy advocates and led to a class-action lawsuit in Illinois in 2015, which ultimately cost the company $650 million.
Equipped with technology like Leyvand’s headgear, Facebook could prevent users from ever forgetting a colleague’s name, serve as a party reminder about a acquaintance’s children, or assist in locating someone at a crowded conference. However, six years later, the company, now known as Meta, hasn’t released a version of that product, and Leyvand has joined Apple to work on its Vision Pro augmented reality glasses.
In recent years, start-ups like Clearview AI and PimEyes have shattered the boundaries of what the public believed was possible by launching face search engines coupled with millions (PimEyes) or even billions (Clearview AI) of photos from the public web. With these tools, accessible to the police via Clearview AI and to the general public through PimEyes, a single snapshot can be used to find other online photos featuring the same face, potentially revealing personal information, social media profiles, or compromising images that individuals would prefer to keep private.
What these start-ups achieved wasn’t a technological breakthrough but rather an ethical one. Tech giants had already developed the ability to recognize unknown faces years earlier but had decided to withhold this most extreme version, which associates a name with a stranger’s face, due to its potential dangers.
Now that the prohibition has been broken, facial recognition technology could become ubiquitous. Already used by the police for solving crimes, authoritarian governments to monitor citizens, and businesses to identify their adversaries, this technology could soon be within everyone’s reach, incorporated into smartphone apps or even augmented reality glasses, ushering in a world free from strangers.
‘We decided to stop’
As early as 2011, a Google engineer unveiled a tool he had been working on that allowed users to search someone’s face and retrieve other online photos of them. Months later, Google Chairman Eric Schmidt revealed during an interview that Google had “built that technology, and we withheld it.”
“As far as I know, it’s the only technology that Google built and, after looking at it, we decided to stop,” Schmidt disclosed.
Intentionally or not, the tech giants also hindered widespread adoption of facial recognition technology by acquiring advanced start-ups that offered it. In 2010, Apple purchased a promising Swedish facial recognition company called Polar Rose. In 2011, Google acquired a U.S.-based face recognition company popular among federal agencies named PittPatt. And in 2012, Facebook bought Israeli firm Face.com. In each instance, the new owners discontinued the acquired companies’ services to external users. The Silicon Valley behemoths were thereby the de facto gatekeepers of the technology’s utilization and availability.
Facebook, Google, and Apple implemented facial recognition technology in what they considered innocuous ways: as a security measure to unlock smartphones, a more efficient method of tagging known friends in photos, and an organizational tool for categorizing smartphone photos based on the people in them.
However, in recent years, smaller and more aggressive companies like Clearview AI and PimEyes have trampled over these barriers. This shift was made possible by the open-source nature of neural network technology, which now forms the foundation of most artificial intelligence software.
Understanding how facial recognition technology progressed can aid us in navigating future advancements in AI, such as image and text generation tools. The power to determine what these technologies can accomplish will increasingly fall into the hands of individuals with even a modest level of tech expertise, who may not adhere to the public’s notions of acceptability.
‘Standing on the shoulders of giants’
How did we reach a point where someone can spot a “hot dad” on a Manhattan street and then attempt to discover their identity and workplace using PimEyes? The simple answer lies in a combination of freely available code shared online, an extensive array of public photos, academic papers elucidating how to piece everything together, and a nonchalant attitude towards privacy laws.
Hoan Ton-That, co-founder of Clearview AI and the driving force behind the company’s technological development, lacked a specialized background in biometrics. Prior to Clearview AI, he had developed Facebook quizzes, iPhone games, and whimsical apps like “Trump Hair,” which made the subject of a photo appear to possess the former president’s distinctive hairstyle.
In his quest to create a groundbreaking and more lucrative app, Ton-That turned to free online resources, including OpenFace, a “face recognition library” developed by a team at Carnegie Mellon University. The code library was accessible on GitHub, accompanied by a warning: “Please use responsibly!”
“We do not support the use of this project in applications that violate privacy and security,” the statement cautioned. “We are using this to help cognitively impaired users sense and understand the world around them.”
While the request was noble, it proved unenforceable. Ton-That managed to get the OpenFace code up and running, but it was far from perfect. He continued to search and experiment, scouring academic literature and code repositories, trying out different methods to see what worked. He metaphorically likened himself to a person strolling through an orchard, sampling the fruits of decades of research, ripe for the plucking and completely unrestricted.
“I wouldn’t have been able to accomplish it if I had to build it from scratch,” Ton-That acknowledged, appreciating the contributions of prominent figures in computer vision and artificial intelligence, such as Geoffrey Hinton, known as the “godfather of A.I.” “I was standing on the shoulders of giants.”
Ton-That continues to innovate. Clearview has developed an augmented reality glasses-compatible version of their app, a more refined manifestation of the face-identifying hat that the Facebook engineering team had rigged up years ago.
The end of anonymity
These augmented reality glasses, priced at $999 and manufactured by Vuzix, connect the wearer to Clearview’s database of 30 billion faces. While Clearview’s A.R. app, which can identify individuals up to 10 feet away, is not yet publicly accessible, the Air Force has provided funding for exploring potential military base applications.
On a fall afternoon, Ton-That demonstrated the glasses to me at his spokesperson’s apartment on the Upper West Side of Manhattan, placing them on and directing his gaze towards me.
“Ooooh, 176 photos,” he remarked. “Aspen Ideas Festival. Kashmir Hill,” he read aloud from the image caption of one of the displayed photos.
Then he handed me the glasses, and I put them on. Although they appeared bulky, they were surprisingly lightweight and fit comfortably. Ton-That mentioned that he had tested other augmented reality glasses but found these to be the best performers. “They have a new version coming,” he mentioned. “They’ll look cooler, more hipster.”
When I looked at Ton-That through the glasses, a green circle materialized around…
Denial of responsibility! Vigour Times is an automatic aggregator of Global media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, and all materials to their authors. For any complaint, please reach us at – [email protected]. We will take necessary action within 24 hours.
