New ACCS evaluation of Yoti age estimation shows shows further accuracy gain

Yoti has scored a compliance win for its facial age estimation (FAE) software. A blog from the age assurance and ID verification company confirms Yoti has been re-evaluated by the Age Check Certification Scheme (ACCS) for FAE on its September 2024 model. The fresh evaluation shows reductions in Mean Absolute Error (MAE) for 18-year-olds – and suggests that the source of test images matters in scoring FAE.

“ACCS now reports our Mean Absolute Error (MAE) for 18 year olds is just 1.05 years, with a Standard Deviation (SD) of just 1.01 years,” writes product marketing director for Yoti, Matt Prendergast. “ACCS first tested Yoti’s September 2020 model in November 2020, reporting the MAE for 18 year olds to be 1.79, demonstrating our continued effort to improve the performance of our model.”

The numbers are noticeably lower than the recent NIST evaluation of Yoti’s September 2024 age estimation model for 18 year olds, which showed an MAE of 2.63 years using visa images captured in Mexico consular offices and an MAE of 2.90 years using application images captured in US immigration offices.

Prendergast acknowledges that NIST testing on FAE models is highly beneficial to the industry. But he says “it is important to understand the NIST test images are not captured on mobile phones and the latest ACCS independent testing shows this makes a big difference to accuracy.”

Since 2018, Yoti has been training its FAE algorithms mainly on images captured with mobile phones, since “the vast majority of demand for digital age checks comes from end users choosing to use their phone to capture their selfie.”

The UK-based biometrics and digital identity company has also conducted its own internal evaluations, which found an MAE for 18 year olds of 1.24 years. Yoti hopes that the ACCS finding coming in lower than its own testing will help build trust among regulators who are still hesitant to embrace age estimation technology.

In terms of age buffers, the ACCS results show a slight decline as the buffer grows. So, while 99.3 percent of 18 years are reliably estimated to be under 25 (a 7 year buffer), 94.2 percent of 18 year olds are reliably estimated to be under 20 – a 2 year buffer.

Still, Prendergast says, “this means a very high percentage of adults can take a selfie and use Yoti FAE to show they are 18 or over. Some may claim it is not accurate, easy to spoof or that biometrics is biased against females or skin tone. We publish results by gender and skin tone which demonstrate there is no material bias.”

The latter point is becoming increasingly crucial to the widespread adoption of age estimation tools. Claims of bias are often leveled at biometric tech, when the numbers from increasingly standardized evaluations show otherwise.

Yoti CEO challenges age assurance critics to beat liveness in court

In a post on LinkedIn, Yoti CEO Robin Tombs offers comment on the upcoming US Supreme Court case between petitioner Free Speech Coalition and respondent Paxton (Texas Attorney General), in which Paxton is pushing for age verification as a requirement for accessing porn – part of controversial law HB 1181.

Tombs pushes back against some of the more obvious misconceptions among “opponents and supporters of the bill” who “reference evidence from Yoti, or about Yoti’s tech, in support of their cases.”

The most egregious? The claim that “it’s easy to circumvent selfies captured for facial age estimation (or to match against ID document faces) by replaying a video or displaying a photo to the camera,” made by, among others, the Centre for Democracy and Technology. Tombs ranks this as a 9 on his scale of falsehoods that qualify as “whoppers.”

“The Texas law requires age assurance to be effective,” he writes. “Without ‘liveness’ tech to detect presentation attacks (PAD) using screen or paper facial images, the face match check or age estimation check would be pointless as the selfie presented to the phone or laptop camera might not be the real user looking into the camera.” Liveness detection, then, deflates that particular argument. And, again, there are stats to prove it.

“Over the last 5 years, iBeta Quality Assurance has evaluated many vendors for PAD liveness effectiveness in accordance with NIST. Level 1 requires a vendor to detect 100 percent of over 500 screen and paper attacks. Level 2 includes more challenging ‘mask’ attacks (costing up to $300) which requires a vendor to detect 99 percent of attacks and reject no more than 15 percent of genuine faces presented to camera.”

“This PAD testing is well understood in the biometrics industry,” Tombs says. “It’s very odd the authors seem unaware of it, or have chosen to ignore it.”

As the court battle over HB 1181 continues, Tombs would like to see NIST called as a witness for the defense – and also suggests that critics of age assurance tech be challenged to beat leading vendors’ PAD/liveness detection using screen and paper attacks. “This type of ‘STEM’ experiment would be cool for children and adults to watch or read about,” Tombs says – keeping quiet on how satisfying it might be for the age verification sector.

Article Topics

Age Check Certification Scheme (ACCS)  |  age estimation  |  biometric liveness detection  |  biometric testing  |  biometrics  |  face biometrics  |  presentation attack detection  |  Yoti