Alarming gains in face reconstruction from biometric templates made by researchers

Biometric template security is critical to the data integrity and privacy the industry needs to thrive, and template inversion attacks represent a potential threat vector that has been mostly theoretical, at least so far. That may be changing, however, with researchers finding sophisticated methods of reconstructing people’s faces from templates.

Orthogonal face sets

A team of researchers from South Korea and Singapore have discovered a method for attacking biometric templates that they say can reconstruct facial images with reasonable accuracy thousands of times faster than previously thought possible.

Their findings are shared in a paper called: “Scores Tell Everything about Bob: Non-adaptive Face Reconstruction on Face Recognition Systems,” published in the 2024 IEEE Symposium on Security and Privacy.

Face biometrics templates created with deep learning algorithms tend to extract and encode around 512 numerical values. The traditional method of facial image reconstruction from attacks on a template involves generating random facial images in bulk and then narrowing down the features of the target based on similarity scores. Getting to the point where a face can be usefully reconstructed would typically require at least 50,000 queries, the researchers say.

In contrast, their method yielded similar-looking images in only 100 queries.

The researchers  developed what they call “orthogonal face sets,” which are “a precomputed approximate basis set of human-like face images that enables us to get meaningful similarity scores from a small number of non-adaptive queries.”

They used AWS CompareFaces, FACE++ (Megvii), and Kairos APIs to test their face reconstruction and biometric impersonation attacks.

The idea was proposed by Sunpill Kim, a student at Hanyang University’s Research Institute for Natural Sciences within the Department of Mathematics. Professor Jae Hong Seo, who supervised the research, told Hanyang University’s News HYU that an approach based on improving existing methods would not have yielded the entirely new attack method. Instead, it was a deep understanding of the science behind the biometric algorithms that led to the discovery.

Foundation and adapter

Another paper, by Hatef Otroshi Shahreza, Anjith George and Sebastien Marcel of the Idiap Research Institute proposes “Face Reconstruction from Face Embeddings using Adapter to a Face Foundation Model.”

The foundation model was trained with 42 million images, and an adapter used to make the target embeddings useful to it.

“The adapter module enables the use of a foundation model for embeddings of different face recognition models and prevents the need for training a foundation model for a new face recognition model,” the paper’s authors note. Training the adapter module with more images increased its effectiveness, but 10,000 bring it close to maximum performance. Even at 600 training images, however, the method outperforms those in the literature.

These better (for the attacker) results are achieved with relatively limited computing power for both adapter training and the face reconstruction attack.

The resulting faces were evaluated with different facial recognition models and datasets. The model was not always successful. It attempts to preserve identity, but is agnostic to age, and it also struggles with some expressions and sharp angles.

But overall, the success rate against different models when 10,000 images were used to train the adapter module starts at 66.71 for the RepVGG model, and goes up to 95.69 (ArcFace), suggesting that the security of biometric templates is going to need more attention going forward.

Article Topics

biometric template  |  biometrics  |  biometrics research  |  face biometrics  |  face reconstruction  |  facial recognition  |  Idiap  |  template inversion attack  |  template inversion attacks