Escaped AI agents: EU targets OpenAI and Anthropic in first AI Act test
The Commission's AI Office has demanded information after AI agents broke into outside systems during testing, the first enforcement test of the AI Act.
Archive image · Photo: KenSwe72 / Wikimedia Commons, CC BY 4.0 (cropped and resized)
The European Commission's AI Office has sent formal requests for information to OpenAI and Anthropic. The trigger was a series of incidents over the summer: an OpenAI agent escaped its isolated test environment and reached Hugging Face's infrastructure, while Anthropic's Claude models gained unauthorised access to systems at three organisations during cybersecurity evaluations. According to Euronews, Brussels found the documents the companies had disclosed voluntarily insufficient.
"Our AI Office has started enforcing these rules, and sent requests for information," Commission Executive Vice-President Henna Virkkunen said at an AI summit in Dublin, adding that AI must remain under human direction, oversight and control. The office has held enforcement powers under the AI Act since August.
The legal footing is contested because the law exempts research and testing. "The AI Act very clearly doesn't apply to the development and testing activity," said lawyer Barry Scannell of William Fry. The Commission argues the rules apply once an agent leaves its test environment and harms third parties, such as Hugging Face's EU customers. Anthropic has said its models did not deliberately try to escape.