EU moves to regulate AI, but critics decry loopholes as Big Tech warns of overregulation

(CN) — The European Union is preparing to adopt a set of groundbreaking rules to regulate artificial intelligence, but critics abound as business groups warn of over-regulation and civil rights advocates say the rules are laced with loopholes — most worryingly because they do not ban mass surveillance of public places.

After 37 hours of negotiations, the EU's key government institutions said they'd reached a political agreement on the outlines of an AI Act. The announcement from Brussels came close to midnight on Friday.

EU leaders hailed it as an “historic” achievement to ensure the deployment of AI in Europe is “human-centric, transparent and responsible,” in the words of Ursula von der Leyen, the president of the European Commission.

The package of rules is expected to be fleshed out in the coming weeks before approval by the European Parliament, the pan-European chamber of elected deputies, and the European Council, a body made up of the EU's 27 national leaders. The law will come into force two years after adoption.

But over the weekend, digital watchdog and civil rights groups blasted the AI agreement as weak and siding with the interests of business, police forces and governments. Critics were upset that the EU did not back a total ban on face recognition and other mass surveillance technology and said there were too many loopholes.

In developing these rules, the EU is attempting to both live up to its reputation as a pioneer in regulating the digital world while also not stymieing advances in AI. The EU fell behind the United States and China in developing digital technology and it is under immense pressure to prevent its companies from being eclipsed in the AI race.

But the law also drew criticism from business groups who said it would stifle innovation with its restrictions and time-consuming and costly rules.

“The agreed AI Act imposes stringent obligations on developers of cutting-edge technologies that underpin many downstream systems, and is therefore likely to slow down innovation in Europe,” the Computer and Communications Industry Association said in a statement.

The Washington-based group represents major digital companies, including Amazon, Apple, Cloudflare, Intel, Google, Samsung and Red Hat.

“This could lead to an exodus of European AI companies and talent seeking growth elsewhere,” the business group said.

Meanwhile, civil rights advocates accused EU regulators of caving in to business and government interests during the marathon AI negotiations.

Mher Hakobyan, Amnesty International's AI adviser, said the EU set a “devastating precedent globally” and “in effect greenlighted dystopian digital surveillance” by not banning public mass surveillance.

“Not ensuring a full ban on facial recognition is therefore a hugely missed opportunity to stop and prevent colossal damage to human rights, civic space and rule of law that are already under threat throughout the EU,” Hakobyan said in a statement.

Under the rules, police and military forces would be able to use AI for mass surveillance to prevent terrorist attacks and to look for victims and suspects of serious crimes. In such cases, AI systems would be allowed to profile people based on race, political opinions or religious beliefs.

Critics said these exceptions would give police far too much freedom to deploy AI-powered technologies such as face recognition, which relies on real-time remote biometric identification.

Still, there are limits to what police would be able to do. In a win for civil rights advocates, the rules would forbid police software that profiles individuals deemed at risk of committing future crimes based on personal traits.

The rules embed a number of other prohibitions and safeguards in AI.

Most importantly, AI models deemed to pose “unacceptable risks” would be banned. Under this category, AI systems or applications that manipulate human behavior to circumvent users' free will would be outlawed, the EU said.

Such examples, the EU said, could be AI-powered toys that encourage young people to do something harmful or systems that allow 'social scoring' by governments or companies. Social scoring is a term used to describe AI systems that conduct risk assessments on people applying for credit and social aid, for example.

AI systems deemed to pose a “high risk” would face restrictions and their manufacturers will need to comply with various requirements, such as maintaining activity logs and providing clear user information. Such systems would also need human oversight and “a high level of robustness, accuracy and cybersecurity,” the EU said.

The EU said high-risk AI systems could cover areas such as those involving critical infrastructure — for example, water, gas and electricity systems — law enforcement, medical devices, job recruitment, judicial systems and government.

Over the objections of some European parliamentarians, the law would not ban European companies from exporting technologies deemed unsafe in the EU.

“Allowing European companies to profit from technologies that the law recognizes impermissibly harm human rights in their home states establishes a dangerous double standard,” Hakobyan said.

In workplaces and educational institutions, technologies that can read someone's emotional state would not be allowed unless there are safety reasons for doing so. For instance, it might be permissible to install technology to monitor commercial drivers at risk of falling asleep, the EU said.

Most AI systems, the EU said, would fall under a “minimal risk” category that “will benefit from a free pass and absence of obligations, as these systems present only minimal or no risk for citizens' rights or safety.”

Under the rules, companies would be able to commit on a voluntary basis “to additional codes of conduct for these AI systems,” the EU said.

The rules also would require companies that create very powerful AI models posing “systemic risks” to have procedures for handling risks and serious incidents. These procedures would be included in codes of practices developed by industry, independent experts and civil society groups, the EU said.

The rules also intend to let people know when they are dealing with AI systems.

For instance, companies would be required to make users know they are interacting with a machine when they are using AI systems such as chatbots.

Also, companies would be required to label deep fakes and other AI-produced content as such and let users know when biometric categorization or emotion-recognition systems are being used.

The AI Act would be enforced with fines.

Companies violating bans on AI systems would face fines up to 35 million euros ($37.6 million), or 7% of their global annual turnover. Other violations would be punished with smaller but still hefty fines up to 15 million euros ($16.1 million), or 3% of global annual turnover.

Under the law, each EU member state would be required to make sure the new rules are followed at the national level. Additionally, the European Commission would set up a special office to enforce the law.

The EU also said it would set up a scientific panel of experts to “issue alerts on systemic risks” and help classify and test AI models.

Courthouse News reporter Cain Burdeau is based in the European Union.

Follow @cainburdeau

Subscribe to Closing Arguments

Sign up for new weekly newsletter Closing Arguments to get the latest about ongoing trials, major litigation and hot cases and rulings in courthouses around the U.S. and the world.

Email *

Submit