Rights groups weigh in on NIST’s evaluation metrics for facial age estimation

Human rights organizations are taking an interest in the metrics used to evaluate age estimation by an American government body.

A call for comments on assessments of age estimation systems using face biometrics from the U.S. National Institute of Standards and Technology has been answered by seven organizations, with 49 comments submitted in total, NIST Biometric Standards and Testing Lead Patrick Grother told Biometric Update in an email. NIST called for feedback on its proposed new track for the Face Recognition Vendor Test (FRVT).

Rights advocacy groups Human Rights Watch and 5Rights Foundation are among those submitting comments, with a letter addressed to Grother and two other prominent NIST scientists. They argue that NIST should calculate accuracy “in discrete years,” with 18 as a boundary due to its common acceptance as the age of majority in most States. They say the use of groupings in a previous report hinders the assessment of algorithm performance.

NIST should also add an indicator for the direction of error, so that the mean error rate, which NIST intends to include, would also indicate whether the algorithm’s mean error judges children as older or younger than their real age.

The groups also ask that NIST disclose information about its testing datasets to “organizations and researchers” that work on children’s and human rights.

Compared to a report NIST produced on age estimation in 2014, the agency plans to employ larger, more diverse datasets with “more demographic dependencies,” reporting for API functions, and enhanced visualizations of the results, according to Grother.

He expressed appreciation for the comments, which he says overall “were highly positive and constructive, went into considerable detail spanning API functionality, analysis and statistics, terminology, use-cases and relevance, and datasets.”

Paravision, which sells facial age estimation software, is pleased to see NIST engaging with the subject, Chief Product Officer Joey Pritikin tells Biometric Update in an email.

“Age verification is rapidly emerging as a key use case within the broader digital identity umbrella, both in response to the need for safe delivery of historically age-gated goods and services as well as in response to and anticipation of a growing number of regulations to protect children in cyberspace,” Pritikin writes. “And within age verification, automated age estimation will be a critical tool alongside face matching, identity document reading, and liveness detection. We expect NIST’s participation will support an environment of transparency, vendor accountability, and stakeholder empowerment, which is what many are striving for with AI broadly and any identity-centric application more specifically.”

Age estimation and age verification have become hot topics in recent months, with new laws and deployments both in the U.S. and abroad.

NIST intends to release an update of the evaluation plan on August 15, with first results expected in late-2023.

Yoti and AVPA call for bias measurement in NIST age estimation assessments

Article Topics

age verification  |  face biometrics  |  Face Recognition Vendor Test (FRVT)  |  NIST  |  Paravision