Face biometrics accuracy identifying mask wearers improves in latest NIST FRVT report

Facial recognition accuracy remains significantly lower when subjects’ faces are covered, but error rates have dropped by around a quarter on average since developers began specifically training their algorithms to meet the challenge, according to a new report from the U.S. National Institute of Standards and Technology (NIST).

The ‘Ongoing Face Recognition Vendor Test (FRVT) Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms’ report evaluates algorithms provided to NIST after mid-March 2020, when numerous biometrics companies began developing facial recognition algorithms with new datasets to accommodate identification of faces partially occluded by masks. The tests were carried out on a 1:1 basis, but NIST says it will carry out 1:N testing in future editions.

Developers do not provide NIST with information about whether their algorithms were developed to work with masks, but the test results indicate that “a number of developers” have adapted their algorithms.

The agency published its finding from tests with algorithms developed before the pandemic earlier this year, and followed it up with an update including more algorithms, but similar results, in October.

NIST received 65 algorithms for the new test, and added assessments for when enrollment is also performed while masked, and results for red and white masks. Cumulative results for 152 algorithms are included in the report.

The algorithms submitted still returned high false non-match rates (FNMR) than on unmasked faces. Some algorithms developed before the pandemic remain among the best-performing for masked faces, some new entries showed significantly improved accuracy, and placed among the top results. Failure rates for the masks covering the largest-area were around 5 percent for all algorithms, compared to 0.3 percent for the most accurate algorithms identifying people without masks. Some algorithms submitted in the new batch, however, still fail to authenticate between 10 and 40 percent of masked faces.

Testing with faces masked during enrollment shows a reduction in FNMR, but false match rates also went up for such systems.

The median FNMR is around 25 percent lower for algorithms submitted since mid-March, NIST says.

Deep Glint and Paravision remain in the top spot for accuracy, while VisionLabs, Hengrui AI and Xforward AI Technology also cracked the top five for accuracy in each of several categories.

Article Topics

accuracy  |  algorithms  |  biometric identification  |  biometric testing  |  biometrics  |  Deep Glint  |  Face Recognition Vendor Test (FRVT)  |  facial recognition  |  NIST  |  Paravision  |  research and development  |  VisionLabs