FB pixel

Go beyond overall biometrics accuracy testing, early and broadly

Fime presents at Authenticate 2023
Categories Biometric R&D  |  Biometrics News
Go beyond overall biometrics accuracy testing, early and broadly

Biometric systems with generally high accuracy can have hidden performance problems in the form of differentials in error rates for different groups of people or based on various other factors. How those differentials are found and addressed in a laboratory setting was explored by Fime Laboratory Biometric Service Line Manager Joel Di Manno in a presentation at the FIDO Alliance’s Authenticate 2023 conference.

In his presentation, “How can bias influence the usage of biometrics?”, Di Manno recounted an experience with a customer seeking quality assurance testing for face biometrics liveness detection.

A particular presentation attack instrument was found to be successful at a rate far beyond the generally strong results. A demographic analysis showed a high frequency of the same skin tone in successful attacks with masks.

There are many possible reasons for this performance disparity, Di Manno explained, including but not limited to the balance of training data. Addressing the errors by improving the performance for the given demographic is a better approach in this case than, for instance, training the system on more masks.

Confirming the suspicion with live subjects, however, would require hiring dozens of people from the affected demographic in a costly and time consuming process, Di Manno points out. Fime’s solution is to use synthetic data, not for an official evaluation, but to test the hypothesis of a demographic weakness in the algorithm.

Once confirmed, the developer analyzed its training data and found a lack of representation for the affected skin tone.

Di Manno proceeded to outline a wide range of factors that can introduce errors, from bias and personal details to environmental challenges and capture device quality. Different factors will create differentials in different modalities, he points out.

An example of an environmental factor introducing errors into fingerprint biometrics is seen in a research paper and subsequent experiment Di Manno summarizes. In a low-humidity environment, the error rate was significantly higher for female subjects than males. In the medium and high-humidity environments, the same difference was not present.

A paper published by Fime in October delves into differentials observed in fingerprint algorithms, finding that the performance one of two was not stable across different environments. This shows, according to Di Manno, the benefits of drilling down into the specific scenarios that can introduce differentials prior to deployment.

He urges developers to take this kind of close look at their products as early as possible in the process, to minimize the real-world impact that different match conditions, from the skin tone of the subject to the humidity level, have on biometric performance.

Article Topics

 |   |   |   |   | 

Latest Biometrics News


IBM loses institutional investor over biometrics sales to Israel

A major Norwegian assets management company Storebrand says it has got rid of its investments in US technology company IBM…


FIDO Alliance introduces passkey Design Guidelines to optimize UX

New guidance on how to implement passkeys for optimal user experience have been published by the FIDO Alliance. FIDO’s Design…


Interpol exec calls for more biometrics sharing to combat cross-border crime surge

Speaking at a recent event in London, Stephen Kavanagh, executive director of police services at Interpol, warned of a “new…


NSW launches digital inclusion strategy consultation on equitable service access

The New South Wales (NSW) government has initiated a call to action for local communities, industries, community organizations, and government…


Could be 25 years before TSA gets facial recognition in all US airports

The Transportation Security Administration (TSA) foresees significant delays in implementing facial recognition across U.S. airports if revenue continues to be…


Single solution for regulating AI unlikely as laws require flexibility and context

There is no more timely topic than the state of AI regulation around the globe, which is exactly what a…


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Most Read This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events