Progress towards unbiased facial recognition seen in EU computer vision challenge

A challenge organized by Human Pose Recovery and Behavior Analysis group as part of the European Conference of Computer Vision 2020 has tested numerous submissions of new, non-discriminatory biometric algorithms in order to dispel biases often seen in facial recognition systems. Winning solutions achieved 99.9 percent accuracy, and low scores in the proposed metrics for bias, which was concluded to be a significant advancement in the development of fairer face biometric methods, announced the University of Catalunya.

The FairFace challenge, initiated and sponsored by AnyVision, aimed to address the issue of discrimination in facial recognition algorithms relating to inaccuracy in identifying ethnic minorities. Sergio Escalera, from the Analysis group organized the challenge with the hope of bettering the biometric accuracy of such systems encompassing all demographic groups.

The 151 participants in the challenge made over 1800 biometric algorithm submissions, working with 152,917 images depicting 6,139 identities. The images were annotated for two protected attributes, gender and skin color, and five others; age group (0 to 34, 35 to 64, 65-plus), head pose (frontal, other), image source (still image, video frame), wearing glasses and a bounding box size. The simulation included significantly more white males than females with dark skin.

“…the adopted dataset was not balanced with respect to different demographic attributes. However, it shows that overall accuracy is not enough when the goal is to build fair facial recognition methods, and that future work on the topic must take into account both accuracy and bias mitigation,” says Julio C. S. Jacques Jr., researcher with the CVC and the Universitat Oberta de Catalunya’s SUNAI Lab and at the Faculty of Computer Science, Multimedia and Telecommunications

The results were published in Computer Vision – ECCV 2020 Workshops.

This post was updated at 10:28am Eastern on January 19, 2021 to note the role of AnyVision in the challenge.

Article Topics

accuracy  |  algorithms  |  bias  |  biometric identification  |  biometric testing  |  biometrics  |  computer vision  |  dataset  |  facial recognition