Innovatrics tops iris matching ranks in 2021 DHS biometrics rally, 2022 to tackle groups

Innovatrics has broken cover to announce it ranked tops for iris matching in the 2021 U.S. Department of Homeland Security (DHS) Science and Technology Directorate (S&T) Biometric Technology Rally. The upcoming rally will test systems’ abilities to deal with mixed crowds where only some people have opted in.

The names of companies entering the rallies are replaced with codenames. As the results of the 2021 rally held last September emerged this spring, Paravision came straight out as the company behind the codename ‘Salt’ as the highest-ranking facial recognition system for matching images. It achieved a 100 percent true identification rate on a range of subjects with images captured by three different acquisition devices.

Innovatrics has revealed it was the ‘Charles’ of the 2021 rally results. And now that we know who Charles is, we went back to the figures as Innovatrics competed against ‘Bayou’ and ‘Hudson.’

Unlike more typical lab testing, the rallies aim to recreate more realistic and demanding situations that the DHS operates in. They are held at the Maryland Test Facility (MdTF). The fourth rally, in 2021, looked at mask-wearing in high throughput situations, compared to without masks. Those being tested also had to pass through or use the biometric equipment themselves, without help.

Biometric matching systems were tested on multiple acquisition devices for face images. For iris testing there was just one piece of hardware for the three matching systems to use (codename ‘Tekoa’).

Even for iris testing there were rounds with and without a face mask.

The 2021 rally had the additional goal that systems be able to identify people with comparable accuracy regardless of their demographics.

The iris technologies were tested on almost 600 people, with breakdowns of results for gender, self-identified race and skin tone assessed via a calibrated color meter device.

For some reason, the iris acquisition device worked better – or participants worked better – when they kept their masks on (481 images submitted from 595 participants with masks on, compared to 466 images submitted from 598 participants when masks were off).

Overall, the three sets of matching software did only a little better with masks off, with less than a percentage point difference for Innovatrics when tested with or without. In fact, in the tests where participants were divided by their self-reported race, all three providers scored higher for matching Asian participants wearing masks that without. For example, for Innovatrics the figures are 100 percent matching focused true identification rate (TIR) for mask wearers (53 out of 53 extracted images) vs 92 percent without masks (46 out of 49 extracted images). However, these are small sample sizes.

Innovatrics led – with or without masks – across all categories. But again, there were common trends. All three systems performed better at iris matching with people of darker skin tones, with accuracy dropping as skin tones became lighter. No details of eye color or lightness are provided. For example, without masks, Innovatrics scored 99.4 percent matching focused TIR among the T1 group, with the darkest skin tones. Then 97.3 and 96.2 for T2 and lightest skin tone group, T3. The competitors had much the same results ranges.

Unlike its two competitors, Innovatrics failed to extract a couple of submitted images.

“Our results show that our new iris recognition algorithm is well-suited for real-life situations it is most commonly used for,” comments Matus Kapusta, Director of ABIS Business Unit at Innovatrics.

These data were shared by permission of the Department of Homeland Security Science and Technology Directorate (DHS S&T). Use of these data in whole or in part does not constitute an endorsement by DHS S&T, the Maryland Test Facility (MdTF), or SAIC.

Biometrics Technology Rally past winners and 2022 theme

Beyond the iris results, the 2021 results revealed slightly more accurate results for lighter skinned volunteers for face matching, and that the majority of errors stem from image acquisition rather than the software-based matching.

Along with the Paravision and Innovatrics announcements for 2021, Corsight AI came out top for facial recognition in 2020, according to Security Journal UK.

There is a whole new challenge for 2022.

“Previous rallies were focused on effectively processing individuals one at a time with and without face masks. However, people often travel in families or groups,” says Arun Vemury, director of S&T’s Biometric and Identity Technology Center, in the call for 2022 entrants.

“Now we plan to assess how well these systems work when identifying individuals within small groups that opt-in to using the system, while avoiding bystanders in a crowded environment who have not opted-in. We believe that this more difficult and realistic scenario will challenge the biometrics industry to innovate for the benefit of travelers and DHS stakeholders.”

The rally is about more than just testing for vendors. They have the opportunity to put their products in front of big wigs from large departments such as DHS and Department of Defense.

The period for entries to the 2022 rally has just closed. Successful applicants will receive conditional acceptance by the end of July and the next rally is planned for 14 to 30 September 2022.

Article Topics

accuracy  |  biometric identification  |  biometric matching  |  Biometric Technology Rally  |  biometric testing  |  biometrics  |  DHS  |  Innovatrics  |  iris biometrics  |  Maryland Test Facility (MdTF)