DHS finds masks cause jump in face biometrics errors for some races

A third of the facial recognition systems tested by the U.S. Department of Homeland Security showed accuracy rates meeting a minimum performance standard of 95 percent matching success across all racial groups. Add face masks, however, and accuracy rates show disparity along racial lines.

Signal reports that at the recent AFCEA 2021 Federal Identity Forum and Expo, DHS Maryland Test Facility Technical Director Yevgeniy Sirotin explained that in the third annual biometric technology rally, the top-performing systems failed to reach the threshold for accuracy for Black and African American subjects wearing masks.

Among maskless white volunteers, slightly over half of the 60 systems tested by the DHS Science and Technology Directorate (S&T) reached the 95 percent threshold. The very best systems had 100 percent match success for people in all self-identified racial groups. None reached 95 percent for Black or African American people wearing masks, however.

With people around the world forced to wear masks in public settings, including the airport environments the scenario test is set up to emulate, higher error rates for passenger identity checks are a practical concern.

The errors observed in identifying darker-skinned volunteers came both in face acquisition and matching, Sirotin said.

DHS S&T Biometric and Identity Technology Center Director Arun Vemury demonstrated the breakdown in face detection with a piece of paper and his virtual background technology.

“There needs to be additional focus on the actual acquisition component of the biometric system,” said Sirotin, according to Signal, “including the quality of the cameras and things like contrast. When you put on a face mask, it really disrupts the local contrasts around the face … especially when you have a contrast of a very bright white mask against maybe a darker skin tone.”

Overall, the top-performing facial recognition system reached 96 percent of masked faces, Federal News Network reports. Adding masks degraded the match rate of the median system from 93 percent to 77 percent.

The 2021 Biometric Technology Rally is set to run next month at the Maryland Test Facility (MdTF).

Also at the AFCEA event, John Stanley, an independent consultant working for the Department of Defense’s Under Secretary for Research and Engineering noted the importance of work on standards provided by the National Institute of Standards and Technology (NIST) for ensuring the interoperability and assessing the relative performance of different biometric systems.

Article Topics

accuracy  |  biometric identification  |  Biometric Technology Rally  |  biometric testing  |  biometrics  |  DHS  |  facial recognition  |  Maryland Test Facility (MdTF)  |  NIST  |  standards