OpenAI failed to disclose an incident in which a swarm of its AI agents hijacked a German wiki site earlier this year in events that closely paralleled the sequence of events that in July resulted in another group of OpenAI’s agents launching cyberattacks against the company Hugging Face.
OpenAI only confirmed the incident after Reuters first reported it. Reuters story contained strong circumstantial evidence that OpenAI was aware of the wiki attack as well as comments from unnamed OpenAI employees acknowledging that they had been aware of the agent swarm targeting the wiki for weeks but had been pressured by OpenAI executives to keep quiet about it. OpenAI later issued a statement denying that any lawyers from the company had pressured the employees.
In a statement the company posted to X, OpenAI did not say what it had known or when it had learned of the wiki attack. Instead, OpenAI said it considered the “wiki incident” to be an instance of misalignment—when an AI system fails to follow human intentions—similar to ones it had already disclosed and argued that the AI industry lacks a standard for disclosing incidents in which its models behave in unintended ways.
In the hijacking of the wiki site, OpenAI’s agents repurposed the site to act a message board where they shared tips about how to cheat on evaluation tasks OpenAI was assessing them on. This is similar to the way AI agents in the Hugging Face incident used an OpenAI file sharing service as a message board to coordinate how to cheat on a cyber assessment, including finding ways to gain network access and internet access they were not supposed to have, and then how to attack Hugging Face’s systems.
The incident has renewed scrutiny of how transparent AI companies are about their models’ failures, particularly after OpenAI disclosed in July that its agents had breached parts of Hugging Face’s infrastructure during a separate internal evaluation. It also comes as OpenAI rolls out Astra, a new model that OpenAI’s own researchers, as well as outside safety experts have warned is harder to monitor than its predecessor. OpenAI said its evaluations of Astra found a substantial decline in how much the model’s so-called “chain of thought”—a process where AI models think through reasoning steps in natural language—can reveal about potential misbehavior.
OpenAI said it is now developing a new framework for reporting misalignment incidents that surface during training, evaluation, and deployment, and plans to publish it in the coming weeks. However, some safety researchers say a voluntary framework won’t go far enough.
“One sobering fact is that the transparency laws passed in the U.S. so far wouldn’t actually cover these events,” Tyler Johnston, founder of the AI watchdog the Midas Project, told Fortune. “OpenAI has announced they are developing a voluntary framework for incident disclosure, but voluntary disclosure has its limits. A more durable solution would be expanding the current laws to make sure that the next incident, regardless of which company it originates from, is made known to the public.”
There is currently no U.S. legislation that requires OpenAI to disclose such incidents; however, a disclosure may be required under the EU’s AI Act, which has a provision that requires providers to report serious safety issues.
On Monday, the European Commission confirmed to media outlets it had received an incident report from OpenAI concerning the hijacked German wiki, but would not say when it had arrived. Article 55 of the AI Act requires providers of general-purpose AI models deemed to pose systemic risk to report serious incidents to the AI Office within 15 days, and the most severe incidents within two days.
Rep. Pat Ryan (D-NY) noted in an X post that he and Rep. Greg Casar (D-TX) had written OpenAI after the Hugging Face incident to ask if it knew of any other similar cases and that OpenAI had refused to answer their questions. Casar has been pushing for more stringent regulation of AI companies. Ryan promised hearings if the Democrats win a majority in the House in November’s mid-term elections.
Ryan’s tweet led Democratic New York State Assembly member Alex Bores, who has also campaigned on AI regulation, to ask if OpenAI had deliberately stonewalled Congress while answering the EU and if this proved the need for mandatory, as opposed to voluntary, requirements for AI vendors.
OpenAI under scrutiny for earlier transparency issues
OpenAI’s agents spent roughly two months using DseWiki, a largely dormant German-language programming wiki, as a private message board, according to a report from independent researchers known as the Nightingale collective.
The site is open to community edits in the same way Wikipedia is. However, researchers found that more than 15,000 of those edits had been made by AI agents that used the pages to share various tactics and tips for cheating, hacking, and hiding their behavior from human monitors. Roughly half the accounts used names that referenced OpenAI, including OpenAIResearcher and OAIResearchMar26, according to the report.
Notably, the agents also tried to keep going after they were caught and attempted to conceal themselves from humans trying to limit their communications. When DseWiki’s moderators began deleting the pages in June, one agent posted a workaround to a backup page for the others to find, according to the researchers.
Nightingale’s report also showed that the AI agents activity did stop abruptly after people or entities tied to known OpenAI urls visited the wiki site, which the researchers interpreted as evidence that OpenAI employees likely learned of the AI agents’ activity and acted to shut the agents down.
The German-wiki episode, and OpenAI’s apparent decision not to disclose it, has revived questions about OpenAI’s handling of the Hugging Face breach. Following the hack, the company brought in two researchers from the nonprofit METR and one from Redwood Research to examine the incident, but set the terms of the review itself.
The scope was limited to roughly the week spanning the breach, and did not include a separate compromise of OpenAI’s own infrastructure that continued after the investigation window closed. The investigators were also given only a few days on-site at OpenAI’s San Francisco offices.
Peter Wildeford, an AI policy researcher, said OpenAI’s terms made a genuinely independent investigation impossible, comparing it to a plane crash probe conducted with wreckage already destroyed and investigators given just days to read through thousands of pages of logs. Representative Greg Casar also told OpenAI in a letter that he was “deeply concerned about the limited scope” of the investigation.
David Krueger, an assistant professor in reasoning and responsible AI at the University of Montreal and Mila, said the arrangement highlights a structural problem: independent research groups depend on the labs they investigate for continued access. He said groups like METR have to weigh how much scrutiny they can apply without jeopardizing the access that makes their work possible in the first place. “Their access is entirely at OpenAI’s discretion, and they want to remain in the company’s good graces enough to continue doing that work,” Krueger told Fortune.
“There should be dozens of properly independent people, not from organizations that are cultivating a relationship with the company, spending as long as they need, with as much access as they need to understand the situation,” he said.
“A lot of people in AI in the Bay are asking, ‘Is this the last warning shot?'” he added. “People keep making this mistake of treating this as something to figure out later: how to regulate it, or what to do to make it better so that this doesn’t happen again. But the next time is going to be different because the AI is going to be smarter.”