Idemia liveness detection tops DHS evaluation

Idemia Public Security has announced it scored the highest biometric accuracy and fairness in an assessment of its liveness detection technology by the U.S. government.

The result is from the Remote Identity Validation Technology Demonstration (RIVTD) run by the Department of Homeland Security’s Science and Technology Directorate (S&T). Idemia’s liveness detection caught all spoof attempts, in each gender, race and skin tone group in the demonstration, while meeting the demanding requirements for user experience satisfaction, according to the company announcement.

Results for RIVTD Track 3: Liveness Detection were anonymized, as in previous tracks, and have not yet been publicly shared by DHS S&T.

The first RIVTD track dealt with ID document authentication, and the second with selfie biometrics matching.

“Our leadership in the RIVTD evaluation reflects our unwavering focus on creating technologies that not only deliver accuracy, but also operate equitably across diverse populations,” says Vincent Bouatou, chief technology officer, Idemia Public Security. “In a rapidly evolving regulatory landscape, fairness is becoming a legal necessity on top of a moral and ethical imperative.”

RIVTD liveness detection assessment results

Track 3 of RIVTD assessed biometric liveness detection in testing by the Biometric & Identity Technology Center.

It evaluated active and passive presentation attack detection systems with what Maryland Test Facility IDSL Technical Director Yevgeniy Sirotin called a “comprehensive set of attack instruments” in a November webinar announcing the results. The two types of PAD were evaluated differently. Passive PAD systems being put through a technology test with previously acquired samples, while active PAD systems went through scenario testing with new samples.

Selfies were taken on both iOS and Android devices chosen by the liveness providers by 661 volunteers, selected to represent a range of demographics.

Sirotin explained the process of the demonstration, which concluded with an experience satisfaction rating.

DHS set a target bona fide presentation classification error rate (BPCER) of 1 percent, meaning they were intended to be “geared for facilitation,” Sirotin says. The benchmark for BPCER was set at below 3 percent.

RIVTD received applications from 8 active subsystems, and selected 6 for evaluation. Fifteen of 17 passive PAD subsystems were chosen. DHS believes the 21 systems chosen are “broadly representative of the state of the art.”

For active PAD, average transaction time was measured and compared with a benchmark of under 30 seconds, and a target benchmark for positive satisfaction rate was set at over 90 percent. For passive PAD, the average run time benchmark was under 5 seconds.

DHS set an attack presentation classification error rate (APCER) of below 3 percent. Non-response errors count towards BPCER but not towards APCER. The agency also measured the maximum BPCER and APCER across smartphones for both, and attack species in the latter case, and the class of presentation attack instruments with the maximum APCER.

The six active PAD systems ranged in length of time taken from just over 22 seconds to nearly 40 seconds, with PAD-A1 and PAD-A4 consistently meeting the benchmark. Transactions on iOS devices were generally faster. Satisfaction ranged from 70 percent up to 92 percent, but only PAD-A1 met the benchmark on both iOS and Android devices, which means it is the Idemia subsystem. PAD-A4 and A6 were close.

BPCER ranged from 3.5 percent 58.6 percent, with no vendors meeting the benchmark. Security, as measured by APCER, was much better. Ten of 12 system combinations meeting the Class threshold for Class A attacks, while 6 of 12 met the threshold for Class B and C PAIs.

PAD-A1 and A3 detected all attacks during the presentation.

BPCER was significantly higher for older users in 10 out of 12 active PAD system combinations, but differentials were not consistently observed for gender, race and skin tone.

Eleven of the 15 passive PAD subsystems – those that used single images rather than 10-second videos — met the 5-second benchmark, and 9 out of 15 met the BPCER benchmark.

PAD-P1 and P9 were successful in blocking all spoof attempts, and only 21 of 45 combinations met the class benchmark, but again, the number of systems that met the benchmark decreased with more sophisticated PAIs. For each different demographic group, either one or two subsystems out of 15 delivered significant differentials.

DHS hopes to gain a better understanding of the impact of biometrics acquisition on passive subsystems with future tests.

Article Topics

biometric liveness detection  |  biometric testing  |  biometrics  |  DHS  |  DHS S&T  |  IDEMIA  |  Idemia Public Security  |  presentation attack detection  |  Remote Identity Validation Technology Demonstration (RIVTD)