UW releases MegaFace facial recognition algorithms benchmarking results

The University of Washington will be releasing initial results from its MegaFace Challenge at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016) on June 30.

The MegaFace Challenge is the world’s first competition aimed at evaluating and improving the performance of face recognition algorithms on a “million person scale”. The MegaFace challenge works to highlight problems in facial recognition that have yet to be fully solved, including identifying the same person at different ages and recognizing someone in different poses.

Previous testing of facial recognition system utilized a smaller number of image samples for evaluation (typically limited to 10,000 or under).The dataset for the MegaFace Challenge, in contrast, leverages one million Flickr images from around the world that are publicly available under a Creative Commons license, representing 690,572 unique individuals.

After collecting these images, the University of Washington’s Graphics and Imaging Laboratory (GRAIL) then challenged facial recognition teams around the world to download the database and see how their algorithms performed when they had to distinguish between a million possible matches.

“We need to test facial recognition on a planetary scale to enable practical applications — testing on a larger scale lets you discover the flaws and successes of recognition algorithms,” said Ira Kemelmacher-Shlizerman, a UW Assistant Professor of Computer Science and the MegaFace challenge’s principal investigator. “We can’t just test it on a very small scale and say it works perfectly.”

Firms that tested their algorithms included Google and N-TechLab. Google’s FaceNet showed the strongest performance on one test, dropping from near-perfect accuracy when confronted with a smaller number of images to 75 percent on the million person test. N-TechLab also demonstrated strong accuracy.

In contrast, the accuracy rates of other algorithms that had performed well at a small scale dropped by much larger percentages to as low as 33 percent accuracy when confronted with the harder task.

“I’d want certainty that my phone can correctly identify me out of a million people — or seven billion — not just 10,000 or so,” added Kemelmacher-Shlizerman.

Results from the first round of analysis has been published as a paper and was co-authored by Kemelmacher-Shlizerman, along with UW Computer Science and Engineering Professor Steve Seitz, undergraduate student and Web developer Evan Brossard and former student Daniel Miller.

According to the university, the MegaFace Challenge is ongoing and still accepting results.

The team’s next steps include assembling a half a million identities, each with a number of photographs, for a dataset that will be used to train facial recognition algorithms. This will help level the playing field and test which algorithms outperform others given the same amount of large scale training data, as most researchers do not have access to image collections as large those maintained by Google or Facebook. The new dataset will be released towards the end of the summer.

More than 300 research groups are participating with the MegaFace Challenge, and the research itself is sponsored by the National Science Foundation, Intel, Samsung, Google, and the University of Washington Animation Research Labs.

Article Topics

algorithms  |  biometrics  |  facial recognition  |  identity verification  |  IEEE  |  MegaFace  |  standards