EU has an innovative new way of fighting against deepfakes

More than half a century ago, American psychologist Paul Ekman wanted to know whether he could tell if a person would commit suicide just by looking at their face.

He set up a projector and spent hundreds of hours examining a 12-minute interview with a psychiatric patient, frame by frame, taking note of each of her slightest expressions and gestures. During the interview, the patient seemed cheerful. Her face, however, told a different story. Ekman would go on to become one of the pioneers in studying microexpressions, minute facial expressions caused by emotional responses that typically last less than half a second.

Today, European law enforcement agencies are trying to apply the same science to crack down on deepfakes, synthetically created videos of people often used for identity theft, fraud and extortion. Facial biometrics software maker Herta Security is helping them achieve this goal.

The Barcelona-based company says it uses an innovative approach to detecting deepfakes. Unlike other deepfake detection tools on the market, it is trying to create technology that’s independent and neutral from the deepfake technology which is evolving at a rapid pace – hopefully making it future-proof.

“We approach [deepfake detection] from a human point of view, based on microexpressions which are not very well imitated by deepfakes nowadays,” says Manuel Pastor, Herta’s chief technology officer. “They don’t blink in a natural way. They don’t smile in a natural way. It’s something that a person cannot really explain.”

Herta is part of a 12-member consortium that aims to develop the software tools that would help European police departments crack down on cybercrime and identity theft. Known as the European Identity Theft Observatory System (EITHOS), the organization aims to be the continent’s central point for online identity theft, gathering universities, law enforcement agencies and companies working on detecting deepfake faces and voices.

Funded by the EU’s Horizon 2020 initiative, EITHOS comes at a crucial time for Europe. A recent Deloitte poll predicted a surge of deepfake financial fraud over the next 12 months while extortion cases using sexual deepfakes, or “deep nudes,” continue to grab headlines.

The initiative is currently developing four tools to support police investigations against identity theft. Aside from deepfake video detection, this includes deepfake audio detection created by Vicomtech, an applied research and development center for AI based in Spain which supplies EITHOS with text and audio analysis software. Other tools analyze social media botnets for phishing and other scams and extract data from online forums where deepfake generators are discussed.

Herta’s tool compares two sets of facial expressions, one from the suspected deepfake video and one from a video showing the real person.

“Involuntary expressions are not controlled by the conscious part of your brain,” says Pastor, ”When a person is speaking naturally, they blink at natural frequencies and they smile in a particular way that cannot be imitated […]. That’s what we are looking for in potentially fake videos.”

Several machine-learning algorithms are involved in the process. To build the technology, Herta is relying on its experience in creating microexpression algorithms for the National Police of Spain. The tool was designed to detect abnormal behavior of victims and suspects during interrogation.

The new anti-deepfake software is currently in beta testing and is being explored by three law enforcement agencies in Sweden, Greece and Spain. The company expects that the product will be available to other police forces in Europe very soon.

“The final target of the project is to create a commercial product that we can sell mainly to [law enforcement agencies] in Europe, but also outside the European Union,” says Pastor.

According to EITHOS’ three-year plan, the tools should be ready in 2025.

The project was launched in 2022 and is coordinated by the Centre for Research and Technology Hellas (CERTH) in Thessaloniki, Greece. Other participants are Italian company Engineering, the Center for Security Studies (KEMEA) in Greece and the Public Safety Communications Europe (PSCE).

Aside from developing technology tools for law enforcement, another task for the consortium is raising public awareness about identity theft and the safety of personal data. Educational institutions, such as the University of Bologna, The Cyber and Data Security Lab (CDSL) at the Vrije Universiteit Brussel and the Technical University of Madrid are working on studying different aspects of identity theft, including social, psychological and legal.

“Victims of identity theft don’t go to the police very frequently,” says Pastor.” They keep it secret, and they try to solve it themselves because they feel kind of embarrassed.”

Studies have found that victims of identity theft appear in all social spheres, even among the technologically literate and highly educated. Identity theft often involves blackmailing with fake porn videos which leaves victims hesitant to reach out for help.

“It’s very important that citizens are aware that they have the technical tools and also the support of the [law enforcement agencies] on these particular crimes,” says Pastor.

Article Topics

biometrics  |  cybersecurity  |  deepfake detection  |  European Identity Theft Observatory System (EITHOS)  |  expression recognition  |  face biometrics  |  facial analysis  |  Herta Security  |  identity theft  |  Vicomtech