Facial recognition was thought of as a revolutionary breakthrough when it debuted as a proof of concept in the mid-1960s. Although it was originally a “man-machine” technology with an extremely limited potential back in the day, it truly captured the imagination with futuristic use cases.
Fast forward to today – and the evolution of this biometric identification technique has paved its way toward full automation, ubiquity, and jaw-dropping accuracy. It kicks in every time you unlock an electronic device with your face, cross the border between some countries, or walk by the sensors of a modern employee attendance control system. In light of the COVID-19 outbreak, the tech is superseding tickets as a touchless way to enter stadiums and enjoy sporting events in New York and Los Angeles.
Facial recognition is also a powerful instrument for law enforcement agencies to track down criminals, and for governments to surveil their citizens. It is a stronghold of the Skynet mass monitoring system deployed in China, with more than 600 million cameras having been installed across the country.
This undoubtedly useful technology is confronted with quite a few challenges. Despite the whole dynamic progress, its algorithms are prone to error in complex identification scenarios. The fact that people are wearing face masks in public places due to the coronavirus emergency is one such stumbling block.
Furthermore, malicious actors are stepping up their efforts to dupe facial recognition systems and thereby gain unauthorized access to devices and facilities. For instance, the so-called “deepfakes” hinging on artificial intelligence (AI) allow criminals to bypass facial recognition systems in a snap.
In response to these caveats, experts are busy taking biometric mechanisms to the next level, and they have had some success.
Don a Face Mask and Be Recognized Regardless
If you think a face mask can prevent surveillance cameras from identifying you, some of today’s cutting-edge facial recognition systems can prove you wrong. Scientists have given these systems a boost substantial enough to recognize individuals with partially covered faces. The identification accuracy reaches – and often exceeds – a whopping 90% in situations where only half a face is visible.
No matter how bizarre it may sound, the COVID-19 crisis has become the driving force of these improvements. Since more people are wearing surgical masks and respirators when outdoors, video monitoring systems need an overhaul to tackle this challenge.
The coronavirus reality motivated China’s technology giants such as SenseTime and Minivision to rush headlong into the commercial deployment of mechanisms for recognizing faces in such scenarios. Not only can the new algorithms identify people with masks on their faces, but they can also pinpoint those wearing scarves, glasses, caps, and fake beards.
The Experimental Foundation of Advanced Facial Recognition
Facial screening techniques that use limited datasets have been around for quite some time. A 2017 study by a team of Stanford University researchers was one of the earliest initiatives in this area. Postdoctoral fellow Amarjot Singh and his colleagues made the first breakthrough in disguised face identification (DFI).
The traditional facial recognition logic revolves around identifying several key points on a human face. When combined, they form a visual pattern that can be uniquely attributed to a specific individual. These markers are mainly concentrated around the eyes, nose, and lips. To enable successful recognition when the bottom part of a face is hidden, the scientists used a larger number of key points in the areas that remain visible, namely the eyes and nose.
The DFI system relies on what is called the “spatial fusion convolutional network” to spot 14 key points on a photo of one’s face. According to Singh, the accuracy of this approach can fluctuate considerably depending on the disguise and the peculiarities of the background behind the person.
As time went by, more groundbreaking research surfaced in this realm and took the efficiency of facial recognition further. In May 2019, analysts from the University of Bradford headed by Professor Hassan Ugail released a whitepaper reflecting their findings on this matter. They were able to achieve a 90% identification success rate with just the bottom half of a face visible.
Moreover, the results were just as promising based on a scan of both the eyes and nose. In the experiment, three-quarter faces could be identified with close to 100% accuracy. However, individual facial areas such as the cheek, mouth, nose, or forehead returned much lower recognition rates.
China takes it beyond theory
In February 2020, the aforementioned Chinese AI company SenseTime pioneered in the real-world implementation of a proprietary technology that detects a total of 240 reference points on an individual’s face. It can successfully find a match based on parts of a face that remain uncovered. Essentially, the key points around the eyes alone can suffice to capture a uniquely identifiable pattern.
The technology masterminded by SenseTime is being currently used to enhance employee attendance systems in offices. The same goes for software by an ambitious Beijing-based AI startup called FaceGo. That being said, workplace monitoring could be a test run before the large-scale adoption of these solutions.
Minivision, one more Chinese tech company with a decent track record of providing AI and IoT services, followed in the footsteps of SenseTime by launching a facial recognition system that successfully identifies people wearing masks. This move was originally a response to COVID-19 infection spikes in some residential communities across the country.
Minivision’s algorithms are primarily focused on screening and processing key points around the eyes. The tech has been deployed in automated gate lock systems that control citizens’ movements in areas hit the hardest by the pandemic.
There is a caveat, though. According to Hu Jianguo, President of the company’s AI research division, the mechanism is unlikely to yield proper results if deployed in larger communities. The reason is that false positives would be imminent due to a greater number of similar eye patterns being encountered down the line.
No mask, No entry
Believe it or not, facial recognition also works the other way around. It can determine if something is missing on a face. Since not wearing a mask in a public place is now deemed as a detrimental exercise, tagging people who do not comply with the rules is an important disease prevention measure enforced by authorities.
Baidu, another technology heavyweight headquartered in Beijing, developed an open-source facial screening model that leverages AI to pinpoint individuals who don’t don masks. Organizations can adapt this framework to develop tools that fit their specific ecosystems and trigger an alert whenever they identify someone whose negligence puts people nearby at risk.
Apple is Working on “Subepidermal Imaging” Technology
A surgical mask is not the only obstacle to recognizing a person. The “evil twin” phenomenon is another long-standing roadblock in this domain. It makes itself felt when a biometric system cannot accurately distinguish between similar-looking faces, as is the case with twins and siblings.
Apple appears to be thinking out of the box to solve this problem and improve its Face ID feature. In July 2020, it reportedly got a patent for an unprecedented tech that relies on vein matching to achieve a higher success rate in “difficult biometric authentication cases.”
Because the pattern of blood vessels underneath the skin of a face is unique for every individual, this type of imaging leaves hardly any room for error and will likely eliminate the “evil twin” issue for good. This approach will involve machine learning models to analyze data captured by sensors built into Apple’s devices.
Time will tell whether this technology goes mainstream, but it is certainly promising. Not only is it a huge step towards password-less authentication, but it can also become a major evolutionary leap in facial recognition as a whole.
Although facial recognition might have negative implications in contexts like controversial surveillance and encroachment on human rights in totalitarian regimes, it is undoubtedly a useful technology that automates numerous facets of people’s lives. Its future definitely holds new fascinating breakthroughs. Hopefully, though, the likes of China’s Skynet will not live up to their famous movie prototype.
About the Author:
David Balaban is a computer security researcher with over 17 years of experience in malware analysis and antivirus software evaluation. David runs MacSecurity.net and Privacy-PC.com projects that present expert opinions on contemporary information security matters, including social engineering, malware, penetration testing, threat intelligence, online privacy, and white hat hacking. David has a strong malware troubleshooting background, with the recent focus on ransomware countermeasures. https://www.linkedin.com/in/david-balaban/
