FB pixel

NIST testing shows accuracy of face biometric algorithms developed pre-pandemic lowered by masks

 

facial recognition can identify people wearing masks

Biometric facial recognition algorithms developed since the beginning of the COVID-19 pandemic may be able to identify people wearing masks better than legacy algorithms, but even the best ones developed previously do so “with great difficulty” according to the latest testing from the U.S. National Institute of Standards and Technology.

NIST tested 89 commercial facial recognition algorithms for the ‘Ongoing Face Recognition Vendor Test (FRVT) Part 6A: Face recognition accuracy with masks using pre-COVID-19 algorithms.’ In testing with digitally applied face masks, even the best algorithms had error rates between 5 and 50 percent in one-to-one matching.

The research team came up with nine mask variants in black or the blue of surgical masks, some just covering the ‘wearer’s’ mouth and nose, while some cover the whole lower face.

NIST Computer Scientist Mei Ngan, one of the report’s authors, says the test was motivated by the pandemic, and the organization plans to test new algorithms developed with masks in mind later in the summer.

“We can draw a few broad conclusions from the results, but there are caveats,” Ngan notes. “None of these algorithms were designed to handle face masks, and the masks we used are digital creations, not the real thing.”

The accuracy of the algorithms declined substantially in every case. While the top performing algorithm with unmasked faces failed about 0.3 percent of the time, their failure rate was closer to 5 percent for faces occluded with the digital masks. Many “otherwise competent” algorithms failed between 20 and 50 percent of the time. Masked images also caused algorithms to be unable to process the face at all, due to a “failure to enroll or template.”

How much of the wearer’s nose is exposed greatly affects accuracy, according to comparisons NIST did with low, medium, and high levels of nose coverage. The shape and color of the mask also make significant differences, with blue masks or more rounded ones degrading performance less. Time and resource constraints made NIST unable to test the effect of color in great depth.

More encouragingly, while false negatives increased, false positives were stable, or even declined somewhat.

DHS recently expressed concern that its facial recognition systems may be less effective with people wearing masks in a bulletin. Companies claiming to have developed algorithms that can handle masks with high accuracy include Innovatrics and SAFR, while other like FacePhi have developed algorithms based on periocular biometrics.

Future tests, in addition to evaluating newer algorithms, will also test for one-to-many matches and other variations to broaden the insights provided.

Ngan says that accuracy is expected to improve, but that the results are consistent with pervious FRVT evaluations.

“Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment,” she says.

Article Topics

 |   |   |   |   |   |   | 

Latest Biometrics News

 

Findynet funding development of six digital wallet solutions

Finnish public-private cooperative Findynet has announced it will award 60,000 euros (US$69,200) to six digital wallet vendors to help translate…

 

Patchwork of age check, online safety legislation grows across US

As the U.S. waits for the Supreme Court’s opinion on the Texas case of Paxton v. Free Speech Coalition, which…

 

AVPA laud findings from age assurance tech trial

The Age Verification Providers Association (AVPA), and several of its members, have welcomed the publication of preliminary findings from the…

 

Sri Lanka to launch govt API policies and guidelines

Sri Lanka’s government, in the wake of its digital economy drive, is gearing up to release application programming interface (API)…

 

Netherlands’ asylum seeker ID cards from Idemia use vertical ICAO format

The Netherlands will introduce new identity documents for asylum seekers Idemia Smart Identity, compliant with the ICAO specification for vertical…

 

Zenoo integrates Trinsic, Sumsub for advanced digital ID onboarding options

Onboarding and compliance orchestration engine provider Zenoo has formed a pair of partnerships to give its customers a broader range…

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Biometric Market Analysis

Most Viewed This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events