iMARS workshop reveals progress and incident growth in face image morphing

Face image morphing is a significant threat to face biometrics applications such as border crossings and identification.

Since 2020, Europe has had a project dedicated to this threat called iMARS, focused on uncovering document fraud and image manipulation, particularly morphed face images. The research project, funded by the European Commission through the Horizon 2020 program, has been working on developing algorithms for morphing attack detection (MAD) and other fraud detection tools for identity verification software.

iMARS recently held its second workshop where it presented the latest findings on face morphing, highlighting potential technical solutions and regulatory risks connected to processing facial data.

The dangers of face image morphing are not as simple as illegal border crossing, says Frøy Løvåsdal, a senior adviser at the National Police Directorate of Norway.

“We’re also concerned that morphing will facilitate other types of crimes such as labor crime, modern-day slavery, organized and serious crime, and terrorism,” she adds.

Løvåsdal presented research on how practitioners detect morphed biometric data, demonstrating that algorithms play an invaluable role but humans will still need to be trained to detect morphing.

Enhancing biometric algorithms and training humans, however, may not be enough. Many countries are still issuing passports with 10 years of validity making it more challenging to detect whether a person’s photograph looks different due to morphing or simply because of the aging process.

“There’s been tremendous progress in the algorithm development. But when it’s approaching a 10-year passport […], the aging really makes things a lot more difficult not just for humans, but also for algorithms,” says Løvåsdal.

The spread of face image morphing is likely to change how countries issue passports. Many countries still allow citizens to submit their own photo to the passport office. However, allowing physical photographs to be submitted allows ample possibilities for manipulation while digital submission also carries its own set of security risks, according to Luuk Spreeuwers, associate professor at the University of Twente in the Netherlands.

“The most desired thing is to use supervised digital ports of acquisition on the spot where you do the application. So this is live enrollment,” says Spreeuwers.

Some countries including Germany are already enabling citizens to take passport photos within municipal government buildings.

For now, detected instances of face morphing still seem low. iMARS conducted a survey among 20 agencies from 19 countries, including passport offices, border crossings, federal police, ministries and others. The questionnaire is still open and organizations are invited to add their input. Around half so far said that they have never discovered a morphing attack, while the other half stated that they had encountered between one and five cases. The majority were discovered manually, i.e. by a human.

In comparison, some agencies recorded over a hundred cases of other types of image manipulation.

Nevertheless, the bad guys are getting better at using AI to manipulate photos. To catch them, law enforcement agencies will need to use special forensic investigation methods for MAD.

Unlike more traditional biometric presentation attack detection methods which reject manipulated images, the main purpose of forensic methods is to find evidence that an image was manipulated and catch the criminal responsible, Spreeuwers explained in a presentation on detecting morphing in criminal investigations.

“We can use image forensics and different algorithms to visualize artifacts or to visualize the manipulated areas in the images and we can try to reconstruct the criminal’s face,” he says.

The final discussions of iMARS workshops were reserved for legal and ethical issues.

Even though iMARS’ technology should not be considered high-risk, meaningful human oversight is very important, says Abdullah Elbi, a researcher at KU Leuven Center for IT & IP Law, a renowned research university in Belgium and Europe.

The tools and technology developed by the organization will have to be compliant with a host of legal regulations, including the General Data Protection Regulation (GDPR) and the EU’s AI Act. Standards are also under development, including those from ISO and other standardization bodies.

The cost of retraining algorithms or changing systems is high. To avoid this risk, iMARS and its partners, as well as other stakeholders working on similar systems should apply the highest standards such as those associated with high-risk systems, says Elbi.

The organization is examining the potential integration into large-scale IT systems connected to border control such as the Entry and Exit System (EES). To achieve that, iMARS has been developing guidelines and best practices, naming an independent ethical advisor who has conducted two impact assessments.

iMARS’ work is not over. Although the end of the project is planned for August 2024, the organization will ask for an extension, says Claude Bauzou, senior product manager at Idemia.

Meanwhile, the organization has been trying to find out whether global agencies would be open to testing its morphing attack detection tools. Out of 19 agencies surveyed, 15 said yes or maybe to testing iMARS algorithms.

Article Topics

biometrics  |  biometrics research  |  EAB  |  European Association for Biometrics  |  face biometrics  |  face morphing  |  iMARS  |  spoof detection