TLDR: Researchers have developed a new method for creating ‘adversarial faces’ that can fool face recognition systems. Unlike previous methods that rely on many queries or modifying existing images, this approach generates entirely new faces by understanding how face attributes are organized in the system’s internal feature space. It’s highly effective, requires very few queries, and can even create faces that impersonate someone while appearing to have different attributes, posing significant security risks, as demonstrated by successfully bypassing verification on popular dating apps.
Face recognition systems (FRSs) are increasingly common, from unlocking our phones to securing buildings and airports. While these systems offer convenience, they also present significant security and privacy challenges. A new research paper introduces a groundbreaking method for generating ‘adversarial faces’ – synthetic facial images that look different to humans but are recognized as a target identity by FRSs.
Traditional methods for creating these deceptive images often rely on iterative optimization, which means they need to repeatedly query the target FRS to fine-tune the adversarial image. This process can be cumbersome, especially for commercial systems where repeated queries are difficult or impossible. These older methods also frequently depend on ‘transferability,’ where an attack designed for one system might work on another, or require open-source surrogate models to mimic the target system.
The novel approach, detailed in the paper “Non-Adaptive Adversarial Face Generation”, takes a different route. Instead of iterative adjustments, it leverages the inherent structural characteristics of the FRS’s feature space. The researchers discovered that individuals sharing certain attributes, like gender or race, tend to form distinct ‘attributed subspheres’ within this feature space. By understanding and utilizing these subspheres, their method can generate adversarial faces with remarkable efficiency.
A key advantage of this new technique is its ‘non-adaptiveness’ and the incredibly small number of queries it requires. Unlike methods that need continuous feedback from the target system, this approach can succeed with just a single non-adaptive query consisting of only 100 face images. This eliminates the need for transferability or surrogate models, making it highly practical for attacking real-world commercial FRSs.
The effectiveness of this method is striking. It achieved a success rate of over 93% against Amazon Web Services’ (AWS) CompareFaces API at its default threshold. Even when tested against a stricter threshold of 0.99, recommended by Amazon for law-enforcement use cases, the attack still achieved success rates of up to 13.7%.
What makes this method particularly concerning is its ability to deliberately produce adversarial faces that not only impersonate a target identity but also exhibit high-level attributes chosen by the attacker. For instance, an adversarial face could be generated to impersonate a specific person while appearing to be of a different gender or race. This is a fundamental shift from many existing attacks that merely perturb a given image.
The real-world implications are significant. The researchers successfully registered adversarial faces, generated from their own selfies, on globally popular dating applications like Bumble, Badoo, and Tinder. These deceptive accounts passed the platforms’ real-time face verification processes and received official verification badges. This demonstrates the potential for such attacks to be exploited in harmful scenarios, such as romance scams, where identity deception poses serious risks to individuals’ emotional and financial security.
The paper also discusses potential mitigations. One straightforward defense would be for FRSs to return only binary decisions (e.g., “accept” or “reject”) instead of confidence scores. However, this might conflict with regulations like the EU AI Act and GDPR, which mandate a ‘right to explanation’ often provided via confidence scores. Other potential defenses involve adding noise to returned scores or increasing the system’s decision threshold, though these come with their own trade-offs in terms of accuracy and computational cost.
Also Read:
- Beyond the Smile: Uncovering Hidden Biases in AI Emotion Recognition
- Understanding Attribute Impact on Face Recognition Model Behavior
In conclusion, this research introduces a novel, non-iterative, and non-adaptive adversarial attack that exploits the intrinsic structure of face recognition system feature spaces. It challenges conventional assumptions about what is necessary for attack success and highlights a significant vulnerability in current FRS technology, urging further study into more robust training methods.
