iBeta expands biometrics testing to demographic bias

A new biometrics testing service has been launched by iBeta Quality Assurance to help developers ensure their systems work well for all demographic groups.

The new biometric demographic bias test is based on the ISO/IEC 19795-10 standard, which addresses “Quantifying biometric system performance variation across demographic groups.” The completed version of the standard was first published last November.

“A 19795-10 conformance test can be considered as an expansion to our 19795-2 performance testing with additional subjects collected to expand the size of the demographic bins,” explains iBeta Deputy Director of Biometrics and Senior Biometrics Subject Matter Expert David Yambay. “Our focus is on the demographic categories of age, gender, and skin tone (currently using the Monk scale).

“After the biometric data of the subjects are tested, we will be able to analyze the results by each demographic bin after normalizing for differences in subjects per bin and determine whether the coverage intervals for each bin are comparable to provide the bias analysis for the solution.”

iBeta was accredited to test against the 19795-10 standard during its standard biannual audit under NIST’s National Voluntary Laboratory Accreditation Program (NVLAP).

“The increase in usage of biometric technologies through the governmental and commercial sectors has caused the need to question whether a technology is functioning fairly for all individuals,” comments iBeta Director of Biometrics Ryan Borgstrom. “This standard allows providers to be able to showcase demographic fairness and equitability in their solutions through the establishment of metrics and best practices.”

iBeta is also accredited for testing by the DEA, Mastercard, Google/Android and the FIDO Alliance.

Representatives from iBeta provided insights used in Biometric Update and Goode Intelligence’s combined “2025 Face Liveness Market Analysis & Buyer’s Guide.”

Bias evaluation evolution continues

Yambay’s observation that the test currently uses the Monk scale alludes to the 10-shade skin tone scale that is the biometrics industry standard since it was developed in 2022 to provide better nuance with darker shades of skin than the Fitzpatrick Skin Type standard. Work to further refine measurements of skin tone has continued, with Colorimetric Skin Tone (CST) proposed in 2024, and now a new method proposed by researchers.

A team of four researchers associated with INESC TEC and the Fraunhofer Institute have proposed using “ethnicity labels as a continuous variable instead of a discrete value” in their paper titled “Beyond black and white: A more nuanced approach to facial recognition with continuous ethnicity labels.”

They note that “not all identities from one ethnicity contribute equally to the balance of the dataset; thus, having the same number of identities per ethnicity does not represent a balanced dataset.”

The research focused on how to train facial recognition models to avoid a disproportionate number of errors with certain skin tones, but has implications for testing as well. This line of research will be followed by biometrics testing authorities and laboratories, just as the early results from demographic bias evaluations of facial recognition models will be watched closely by the industry as a whole.

Article Topics

accuracy  |  biometric testing  |  biometric-bias  |  biometrics  |  demographic fairness  |  iBeta  |  ISO 19795-10:2024