FB pixel

Face biometrics vendors Oosto and FaceTec identify limitations to NIST FRVTs

Face biometrics vendors Oosto and FaceTec identify limitations to NIST FRVTs

The leading benchmark for face biometrics is not well-suited to evaluating technology for video surveillance use cases, Oosto argues in an article posted on its website.

It is just the latest example of a vendor pointing out the limitations of the highly-regarded Face Recognition Vendor Test (FRVT) from the U.S. National Institute of Standards and Technology, following a similar line of recent commentary from FaceTec.

In ‘5 Limitations of NIST’s FRVT Testing for Video Surveillance,’ Oosto CMO Dean Nicolls begins by explaining what the test is and the importance of NIST as “an independent body that is testing third-party software – a la Consumer Reports or JD Power – and providing industry performance benchmarks.”

Enterprises should be aware, however, that the FRVT is not applicable to video surveillance software, he argues.

Nicolls points out that the test is intended for forensic applications, rather than real-time identification. Function in real-world conditions is also critical for video surveillance, but even the ‘wild’ category in NIST’s test is not equivalent to the kind of images processed by video surveillance systems.

“When I use the term ‘in the wild,’ this is different from NIST’s definition of ‘wild’ where people are not necessarily posing or cooperating for a photo,” Nicolls writes. “NIST’s wild images may be blurry, or of low quality, but the images are still relatively clear, the camera is generally at face level, and the lighting is good.”

FRVT also does not evaluate the ability to pick an individual out of a crowd, consider video camera quality, or test images captured at extreme angles.

The purpose of the post is not to criticize NIST, Nicolls insists, but rather to help enterprises and technology buyers understand the limits of the influential benchmark.

Oosto encourages NIST to create a new testing methodology to handle the particular conditions and requirements of facial recognition in video surveillance solutions.

Echoes of FaceTec PAD commentary

The argument generally tracks with arguments made by FaceTec in its ‘NIST FRVT-PAD Commentary’ submitted by the company in late-April.

The commentary was submitted to NIST in response to a request for comments on its proposed FRVT-PAD (presentation attack detection) framework. While addressing the PAD assessment proposal, the commentary makes points similar to Oosto’s in that they suggest a range of technologies the test will not adequately address.

FaceTec says that the PAD standard proposed as a starting point “has been outdated for years,” referring to the ISO/IEC 30107-3 publication in 2017.

Shortly after the commentary was published, FaceTec announced the development of a 3D face biometrics model that it said delivers a step-change in accuracy, with a false acceptance rate of 1 in 125 million. The algorithm cannot be tested as part of the FRVT, however, which only uses 2D images.

That limitation means that NIST will be unable to assess Apple’s Face ID, as well as FaceTec’s biometrics.

Defense against deepfake video injections should also be included in any test, FaceTec argues. The commentary goes on to explain bypass attacks, and other vulnerabilities, including in-device sensors.

Article Topics

 |   |   |   |   |   |   |   | 

Latest Biometrics News


Smile ID surges past 150M identity verifications completed

Selfie biometrics provider Smile ID (formally Smile Identity) has achieved the completion of 150 million identity verifications – five months…


Incode age assurance gets stamp of approval from ACCS

Incode Technologies has announced that their Incode Identity Platform product has received certification under the Age Check Certification Scheme (ACCS)….


Arana launches police biometrics app for HID scanner

Arana Security has launched a new mobile app for mobile biometric fingerprint readers from HID used by law enforcement. The…


Fraud hammers online services, drives AI ambivalence

Fraud rates are spiking just like temperatures in many parts of the world. Global identity verification companies Sumsub, AuthenticID and…


Contractor distances self from biometric device failures in South Africa elections

A Johannesburg-based company, Ren-Form, which supplied biometric hardware to the Electoral Commission of South Africa (IEC) says it is not…


Online age verification requirements in US legislation raise thorny problems

More than a dozen bills have been introduced in the current U.S. Congress that, if enacted, would increase protections for…


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Most Read This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events