GSA equity test shows selfie biometrics effective, if you pick the right vendor

Identity verification systems based on selfie biometrics can be effective, with strong performance across demographic groups, but results may vary dramatically depending on which vendor’s technology is used.

This is the key finding of an equity study organized by the U.S. General Services Administration to lay the groundwork for the adoption of face biometrics as an authentication method for Login.gov that meets IAL2.

The assessment of remote identity verification with selfie biometrics was carried out by a team of researchers including power couple Michael and Stephanie Schuckers and others affiliated with Clarkson University, the GSA and DataedX Group. They tested the ID verification systems selected with nearly 4,000 volunteers representing a broad range of demographics.

Two of the vendors assessed were found to deliver equitable results across all demographics, while two were found to perform poorly with one or more demographic and the other delivered an unacceptably high false negative rate (FNR) of 53.1 percent, plus or minus 8.2 percent.

The five vendors assessed are identified in the study as Dingo, Hedgehog, Wombat, Marmot, and Badger. Hedgehog was the vendor which failed to pass half of legitimate attempts.

GSA revealed in a 2023 privacy impact assessment that it was considering TransUnion, Socure, Jumio, LexisNexis, Incode and red violet, one of which seems to have been dropped along the way.

Marmot scored the lowest FNR, at 10.5 percent, just ahead of Badger, at 14.3 percent. In addition to having the best overall performance, each returned similar results across all demographic groups. Dingo worked less well for people with darker skin, and Wombat had a significantly lower FNR for the Asian American and Pacific Islander group.

The final conclusion of the researchers is that the study confirms the need to evaluate performance across demographic groups for a complete understanding of how a given system for remote identity verification with selfie biometrics works.

A GSA spokesperson says the results will be shared in a final, peer-reviewed report during 2025.

Article Topics

biometric testing  |  biometrics  |  GSA  |  identity verification  |  remote identity proofing  |  selfie biometrics