Measuring liveness across datasets helps defend complex real-world attacks: researchers

A new research paper evaluates liveness detection models for biometric samples, focusing on performance in cross-database scenarios. Titled “Cross-Database Liveness Detection: Insights from Comparative Biometric Analysis,” the paper was presented in November 2023 in Lviv, Ukraine, by authors from Kharkiv University, Marche Polytechnic University and the University of Macerata in Italy.

Liveness detection in cross-database scenarios, the paper says, is “a test paradigm notorious for its complexity and real-world relevance.”

“In an era where biometric security serves as a keystone of modern identity verification systems, ensuring the authenticity of these biometric samples is paramount,” it reads. The authors believe effective liveness detection that can differentiate between genuine biometric samples and sophisticated spoofs is an important tool in meeting this challenge.

The authors’ approach takes a deep dive into various liveness detection models’ performance metrics, including half total error rate (HTER), false acceptance rate (FAR), and false rejection rate (FRR). But the true test for liveness detection mechanisms, they say, is how robust and adaptable they are across diverse scenarios. There can be a wide gap between datasets that liveness detection is trained on and the ones on which it is deployed. For biometric systems, the stakes are especially high: a system trained exclusively on one database might perform flawlessly on that particular data, but falter when met with a different spoofing technique or demographic distribution.

The solution, say the authors, is a cross-database testing paradigm.

Datasets account for a variety of tactics and conditions

The research used five distinct datasets for evaluating face presentation attack detection (PAD). The Custom Silicone Mask Attack Dataset (CSMAD), collected by the Idiap Research Institute, consists of facial biometric data from 14 subjects, including bona fide presentations and custom silicone mask attacks. The 3D Mask Attack Database (3DMAD) contains 76,500 frames of 17 individuals, recorded using a Microsoft Kinect sensor for depth in both genuine access and 3D mask spoofing attacks, including real-size masks obtained through ThatsMyFace.com. Idlap also provided the Multispectral-Spoof Face Spoofing Database (MSSpoof) of VIS and near-infrared (NIR) spectrum images from 21 subjects, and the Replay-Attack Database, a 2D facial video database of 1,300 real access and attack attempt video clips from 50 people under various lighting conditions.

Finally, the authors used their own dataset of more than 4,600 2D facial images and videos taken with smartphones or downloaded from the internet, mainly YouTube.

Results underline the necessity of cross-database testing

Of the utilized datasets, 3DMAD yielded the best results in initial testing, demonstrating “impeccable performance across all metrics.” The CSMAD dataset, meanwhile, “posed significant challenges.”

“The variation in performance across datasets underscores the criticality of diverse data representation in training robust liveness detection models,” it says. “While some datasets like 3DMAD show near-perfect results, others like CSMAD reveal potential vulnerabilities. Our findings emphasize the importance of comprehensive evaluations and the necessity of cross-database testing.”

When it came to cross-database testing, “several models exhibiting high efficacy on their native datasets encountered significant challenges when subjected to data from external sources.” This, say the authors, is a red flag showing that models, “if overly tuned or biased towards specific dataset characteristics, may fail to maintain performance parity across broader biometric variations.”

The crux of the paper’s argument is that cross-database testing beyond conventional evaluation methods is necessary to ensure robust and adaptable liveness detection. “By exposing models to an array of biometric datasets, we unearth indispensable insights into their true robustness and generalization prowess, informing more reliable, secure biometric verification systems for the future,” it says. “While our model exhibited commendable performance in certain scenarios, the inconsistencies observed in cross-database testing illuminate the path for future research. The journey towards perfecting liveness detection is ongoing, replete with challenges yet filled with opportunities. As spoofing techniques evolve, so must our defense mechanisms, making this a perpetually dynamic field of study.”

Article Topics

3D liveness detection  |  biometric liveness detection  |  biometrics research  |  Idiap  |  presentation attack detection  |  spoof detection