T-Shirts have become a facial recognition threat, a new study shows how to stop it

Discsussions of biometric presentation attacks typically center around financial fraud attempts, but the increasing use of facial recognition in public has spurred researchers to develop clever ways to dupe the technology to evade security or surveillance. One of the methods that has proven successful in controlled experiments is adversarial images presented on T-shirts. The latest development in this field replaces adversarial images and patterns with face presentation attacks: A  T-shirt with a printed human face is presented to the camera, tricking the facial recognition system into believing it is seeing a real three-dimensional face.

A new paper from a group of researchers at the Darmstadt University of Applied Sciences in Germany shows a way to prevent these cheap and effective presentation attacks.

The researchers tested three widely-used, open-source face detection algorithms, RetinaFace, MTCNN, and dlib, against the T-shirt Face Presentation Attack (TFPA) database. The database contains over 1,600 images taken from 100 different T-shirts, each printed with a facial image.

Eight people were recruited to wear T-shirts with printed faces in different poses, while their images were captured with a RealSense D435, a sophisticated camera capable of capturing depth information for 3D images.

In almost all cases, face detection algorithms picked up the face on the T-shirt. The average estimated detection rate of the three algorithms surpassed 99 percent for all eight poses, the results show. The study also showed that if the attacker hid their face, either by covering it with hands, wearing a face mask, or tilting their head, a face biometrics system would likely return a match with the T-shirt – meaning the attack would succeed.

The success of this presentation attack is worrisome, as T-shirts are easy to make compared to other presentation attack instruments, such as 3D silicone masks. T-shirts can also be hidden under a jacket, meaning they are easier to use in supervised conditions compared to something conspicuous like a paper mask. T-shirt attacks have already been identified by border authorities as a potential risk.

To solve this, the researchers extended the TFPA database with 152 bona fide presentations and proposed a new detection method. The proposed algorithm can be easily combined with traditional presentation attack detection algorithms, according to the research.

“[The] state-of-the-art face and person detectors are combined to analyze the spatial positions of detected faces and persons based on which T-shirt attacks can be reliably detected,” the study explains. “The results show perfect detection performance on the used database.”

The study was published on Arxiv and funded by the European Union’s Horizon 2020 research and innovation program and the German government.

Article Topics

adversarial attack  |  biometrics  |  biometrics research  |  face detection  |  facial recognition  |  presentation attack detection