Yoti and AVPA call for bias measurement in NIST age estimation assessments

The planned assessment of age estimation algorithms by the U.S. National Institute of Standards and Technology should note the distribution of errors, include different age targets, and include a measurement of bias, according to comments from industry stakeholders.

NIST received 49 comments, including from human rights organizations, age estimation providers like Paravision and Yoti, and leading industry group the Age Verification Providers Association (AVPA), in response to a call for feedback on a Face Recognition Vendor Test track in development.

In comments provided to Biometric Update, the AVPA urges NIST to consider releasing some of the data it holds for use in algorithm training, noting that “(a)ccess to training data is a huge barrier to entry in this field.”  The organization also calls on NIST to consider the output of the euCONSENT trials of age interoperable age assurance as one of its 12 points of feedback.

Yoti and the AVPA are united in calling for ages other than the ones specified so far by NIST; 12, 18, 21 and 70. COPPA applies to children aged 13, and an amendment to extend it to 16 years of age is under consideration, the AVPA points out. Rights groups similarly noted the importance of age 18 as a boundary.

The industry stakeholders are also aligned in asking for measurements of differences for people with different skin color and gender.

Yoti’s feedback includes a request for more information on how error rates will be computed, and a recommendation to add a plot based on the ROC metric that shows the challenge age (legal age plus buffer), true pass rate and false pass rate in a single figure.

Rights groups weigh in on NIST’s evaluation metrics for facial age estimation

Article Topics

AVPA  |  biometric age estimation  |  biometric bias  |  biometrics  |  Face Recognition Vendor Test (FRVT)  |  NIST  |  Yoti