Researchers analyze quality assessment algorithms for biometric samples

Researchers have published a new paper analyzing quality assessment algorithms that measure the fairness of captured biometric samples.

The paper titled “Fairness Measures for Biometric Quality Assessment” was published by scientists from the Technical University of Denmark and the Biometrics and Security Research Group in Germany,  which is affiliated with the University of Applied Sciences in Darmstadt and the National Research Center for Applied Cybersecurity (ATHENE).

The quality of the sample has a strong effect on the recognition performance of a biometric system. Low-quality samples are normally discarded. However, quality assessment algorithms sometimes yield different quality scores across demographic groups, which leads to different discard ratios. This is why it is crucial to develop a fairness measure that ensures quality assessment algorithms do not take demographic characteristics into account when assessing sample quality, the researchers write.

The paper delves into several measures for the fairness assessment of biometric quality, including Sample Quality Fairness Rate (SQFR) and  Cubed Sample Quality Fairness Rate (CSQFR).

“In general, if there is a preference of achieving even lower fairness scores for biased scenarios while slightly reducing the score of fairer scenarios, we recommend using a variation of the CSQFR over a variation of the SQFR,” the research says. “A promising CSQFR variant could be the LWM-GC-CSQFR, as it behaves similarly to the Mean-GC-CSQFR and additionally has the property of giving higher weight to lower quality scores.”

“On the other hand, this weighting of the LWM-GC-CSQFR may not be necessary for quality score distributions in the field, as these are unlikely to include edge cases as demonstrated in Figure 4 and therefore the simpler Mean-GC-CSQFR may already be sufficient for quality score distributions in the field,” the paper continues.

Researchers conclude that future work could focus on using SQFR on operational quality assessment algorithms, including captured data in the field such as passport enrolment images, kiosk enrolment images and border control probe images. The results could inform standards work already underway.

Academic and government researchers have also suggested decoupling bias from accuracy in assessments and performed extensive testing as the ecosystem grapples with lingering challenges in understanding how well biometric systems work for different groups of people.

Article Topics

accuracy  |  algorithms  |  biometric data quality  |  biometric-bias  |  biometrics  |  biometrics research  |  demographic fairness