Publication of ISO standard sets up biometric bias tests and measurement

The international standard for measuring biometric bias, or demographic differentials, is now available for purchase and preview from the International Standards Organization.

ISO/IEC 19795-10:2024 is the standard for how to quantify the variations in the performance of biometric systems across different demographic groups. It was given final and unanimous approval a month ago, in a step lauded by influential voices in the biometrics community from TSA Identity Management Capability Manager Jason Lim and Algorithmic Justice League Co-founder and Gender Shades Author Joy Buolamwini to Paravision CEO Doug Aley.

Yevgeniy Sirotin and John Howard of the Maryland Test Facility co-edited the standard. Sirotin notes in a LinkedIn post that the standard could also assist in the formulation of metrics and standards for measuring bias in other AI applications.

Because accurately representing accuracy differentials is an exercise in comparison, it must balance the similarity of the results with their effectiveness. The 19795 standard therefore introduces a number of terms and measures not typically found in biometric performance standards.

Among them are differential performance measure, false negative differential performance, false positive differential performance, comparison score differential measure, mated comparison score differential and non-mated comparison score differential, and aggregate equitability measure.

The standard sets out how to plan and execute evaluations, and then calculate the scores and the uncertainty that goes along with them.

Article Topics

accuracy  |  biometric testing  |  biometric-bias  |  biometrics  |  demographic fairness  |  ISO 19795-10:2024  |  ISO standards  |  standards