FB pixel

All eyes on AI Act exemptions as ban on high-risk AI systems nears

All eyes on AI Act exemptions as ban on high-risk AI systems nears
 

Despite being celebrated as the world’s first comprehensive AI legislation in the world, the European Union’s AI Act has left some questions open. This includes carving out exemptions for the use of otherwise banned AI applications for law enforcement and border control agencies.

On February 2nd, the ban on AI systems that pose “unacceptable risk” will become official. The existence of national security exemptions, however, is raising the question whether the AI rulebook will be able to safeguard rights.

Banned AI applications with “unacceptable risk” levels include biometric categorization systems based on sensitive characteristics, emotion recognition in the workplace and schools, social scoring, predictive policing and applications that manipulate human behavior. The European Association for Biometrics (EAB) organized a talk this week, which invited legal experts, industry stakeholders such as Idemia and representatives from the EU’s AI Office to discuss the use of biometric data in these applications.

“It’s not an absolute prohibition, so it requires a well understanding of rules,” says Abdullah Elbi, legal researcher at the Centre for IT & IP Law (CiTiP) at KU Leuven in Belgium.

The most important thing would be having well-reasoned guidelines from the European Commission, market surveillance authorities and data protection authorities, he adds.

Although the AI rulebook seems to establish standards for AI application, it leaves balancing security and rights protections to EU countries themselves: Governments can decide on whether they can introduce exceptions that would allow real-time remote biometric identification in cases such as very serious crimes, searching for victims or preventing serious threats such as terror attacks.

“There could be some fragmentation in different member states in the EU when it comes to the use of remote biometric identification systems,” says Elbi.

The law enforcement carve-outs have led to criticism from rights groups, which claim they water down protections against potential abuse.

However, Irina Orssich from the EU’s AI Office notes that European member States cannot negotiate aspects of the implementation of the AI Act. Individual countries can introduce stricter regulations than what is required by the AI Act, but they cannot introduce more relaxed regulations, she says.

“You still have a tiny bit of margin in practice because these rules will be enforced by member states authorities, and they indeed might take in practice slightly different stances,” says Orssich, who is one of the co-authors of the AI Act.

The AI Office is attached to the European Commission and tasked with coordinating the implementation of the law among EU member states.

Another exception to the AI Act that is awaiting regulation is the evaluation of AI systems on the ground of compliance.

Providers are required to put their high-risk AI systems, including biometric ones, through a conformity assessment procedure. However, the AI Act leaves the door open for placing on the market or putting into service certain systems that have not undergone conformity assessment in exceptional situations such as public security.

“The market surveillance authority may directly accept that the system be used, but this authorization is only limited – conformity assessment procedures must be thereafter carried out for law enforcement authorities and civil protection authorities,” says Lydia Belkadi, another researcher at KU Leuven Center for IT & IP Law.

There are two different types of procedures, including internal control, which is directly confirmed by the provider. In certain cases, the Act requires the involvement of a third party, a Notified Body which will be designated by EU member states.

To ensure compliance with the AI Act, standardization will play a key role. The European Committee for Standardization and the European Electrotechnical Committee for Standardization are working to make the standards available by the end of 2025.

“Overall, high-risk AI systems are now subject to a more comprehensive system of oversight by different actors in the value chain that includes the providers and deployers. And these requirements are really the center key to respect for fundamental rights,” says Belkadi.

Report identifies France as main force in ‘diluting’ AI Act

The loopholes and national security exemptions in the AI Act are the result of a campaign of several EU member states, led primarily by France, a new investigation has revealed.

The country’s administration, led by President Emmanuel Macron, strategically engineered amendments to the regulation to enable law enforcement and border agencies to bypass the ban on remote biometric identification in public spaces. Countries such as Italy, Hungary, Romania, Sweden, the Czech Republic, Lithuania, Finland and Bulgaria expressed support for France.

The carve-out could, for instance, allow climate demonstrations or political protests to come under biometric surveillance if police have national security concerns, according to a report published by Investigate Europe and The EU Observer. The investigative outfit analyzed more than 100 documents, including those from the Committee of Permanent Representatives (Coreper) meetings. Coreper’s role is to prepare the agenda for EU Council meetings.

France and other countries also lobbied for emotion recognition systems to be permitted for all police forces and immigration and border authorities. Previous reports have highlighted how the loophole could lead to rights abuses. Biometric categorization systems based on sensitive characteristics also received an exemption for the police.

France has been experimenting with AI-based surveillance during the Paris Summer Olympics 2024.

Greece is another country that demanded exclusions from the AI Act bans, lobbying for real-time biometric analysis in public spaces on citizens, refugees and asylum seekers.

The report also touched upon compliance assessments for high-risk AI systems. Thanks to exemptions, companies are allowed to conduct self-certification, which could help them skirt obligations.

While some experts, such as Dutch digital rights lawyer Anton Ekker, believe the exemptions will have little impact in real life, others claim that the largest effects could be on vulnerable populations, which have little power to complain.

“In most cases regulation and oversight … only kicks in after the violation has taken place, they do not protect us before,” says Rosamunde van Brakel, assistant professor at the Vrije Universiteit Brussel (VUB).

Related Posts

Article Topics

 |   |   |   |   |   |   |   |   |   | 

Latest Biometrics News

 

Biometrics connecting ID and payments through digital wallets, apps and passkeys

Biometrics are connecting with payment credentials, whether through numberless credit cards and banking apps or passkeys, as the concrete steps…

 

Reach of Musk, DOGE’s federal data access sets off privacy, security alarms

Led by tech billionaire Elon Musk and a shadowy team believed to be under his control, the United States DOGE…

 

Mobile driver’s licenses on the cusp of ‘major paradigm shift’

More entities have integrated the California mobile driver’s license (mDL) credential for identity verification. Although just 15 states have introduced…

 

Gesture-based age estimation tool BorderAge joins Australia age assurance trial

Australia’s age assurance technology trial is testing the new biometric tool that performs age estimation based on hand gestures. The…

 

European AI compliance project CERTAIN launches

The pan-European project to create AI compliance tools CERTAIN has kicked off its work, with the goal of making European…

 

Signaturit Group acquiring Validated ID for undisclosed sum

Spain-based digital identity and electronic signature provider Validated ID is being acquired by Signaturit Group, a European company offering identity…

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Most Viewed This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events