Facial recognition 20 times more accurate with advances in convolutional neural networks, NIST finds

Facial recognition algorithms can identify matches with error rates as low as 0.2 percent given good quality photos, 20 times better than the top 4 percent error rate recorded only five years ago, according to the latest report from NIST’s Ongoing Facial Recognition Vendor Test (FRVT).

The NIST Interagency Report (NISTIR) 8238 (PDF) shows that algorithms from 28 developers are now more accurate than the top-performing algorithm observed in late 2013, and that impressive gains made from 2010 to 2013 have been outstripped by even greater improvements from 2013 to the technologies NIST has tested in 2018. The improvements are due the integration of or change to deep convolutional neural networks, NIST says.

“As such, face recognition has undergone an industrial revolution, with algorithms increasingly tolerant of poor quality images. Whether the revolution continues or has moved into a more evolutionary phase, further gains can be expected as machine learning architectures further develop, larger datasets are assembled and benchmarks are further utilized.”

The performance of 127 prototype algorithms from 39 commercial vendors and one university (the Shenzhen Institute of Advanced Technology), which NIST says represents “a substantial majority” of the industry, though a tiny minority of academia.

Winners in the Ongoing 2018 FRVT include Camvi in the Wild Image category and Yitu in the two Visa categories.

“The implication that error rates have fallen this far is that end users will need to update their technology,” says NIST Computer Scientist and report co-author Patrick Grother. “The test shows a wholesale uptake by the industry of convolutional neural networks, which didn’t exist five years ago.”

FedScoop reports that NIST is planning to release two more reports on facial recognition accuracy in 2019. One will present the results of 90 additional algorithms from 49 developers, and the other will explore demographic dependencies in the technology.

Bias and accuracy discrepancies are being scrutinized at the government level in the U.S., where eight members of Congress recently sent Amazon CEO Jeff Bezos a letter requesting information on its Rekognition service.

NIST warns facial recognition users that the best algorithms are significantly more accurate than most.

“There remains a very wide spread of capability across the industry,” Grother adds. “This implies you need to properly consider accuracy when you’re selecting new-generation software.”

This story was edited at 10:40 ET on December 4 to amend the incorrect inclusion of previous test results among current results.

Article Topics

algorithms  |  biometrics  |  Camvi Technologies  |  Face Recognition Vendor Test (FRVT)  |  facial recognition  |  neural networks  |  NIST  |  Yitu Technology