It’s a brake test at breakneck speed. This weekend, Anthropic CEO Dario Amodei proposed to “slow the pace” of artificial intelligence (AI) development amid rising safety concerns, a call that was quickly joined by OpenAI’s Sam Altman, Google DeepMind’s Demis Hassabis, and SpaceXAI’s Elon Musk. The new push comes after increasing reports of out-of-control AI and researchers raising alarm about the risks to humanity. We turned to our experts to make sense of these developments and explore what they mean for policy.
Click to jump to an expert analysis:
Graham Brookie: As industry pushes on safety, policymakers need to keep pace
Safa Shahwan Edwards: AI industry can learn from cybersecurity on pacing itself
Kenton Thibaut: A major breakthrough on AI safety is highly unlikely
Trisha Ray: “Pacing the frontier” isn’t enough to stop the next Hugging Face incident
Graham Brookie is the Atlantic Council’s vice president for technology programs and strategy.
As industry pushes on safety, policymakers need to keep pace
The call to “pace the frontier” of AI development and the rapid alignment among AI leaders who are in competition with one another is nothing short of historic.
Zooming out, three things are true. First, the United States—with its frontier labs alongside some European labs and critical supply chain dependencies that make up the AI ecosystem—is in a global, systemic competition that will be determined by technological advantage. AI will increasingly be a determinant for national power, military advantage, and economic growth. The outcome of this contest could also determine whether open societies and open markets remain viable compared to a surveillance-based authoritarian alternative.
Second, winning the competition for AI superiority won’t mean a thing if it leads to some of the worst-case scenarios creeping closer as the technology advances faster than US and allied institutions can manage its compounding and autonomous risks.
Third, while we are in an AI “arms race,” the public remains deeply skeptical, nervous, and pessimistic. Justifiably so. The Atlantic Council Commission on AI, which included innovators and executives from the frontier labs among other sectors, found that trust is a key enabling factor for AI leadership. There is a serious trust deficit amid proliferating AI risk. When frontier model researchers abruptly quit lucrative jobs at the top of their profession and declare that AI may have a 10 percent (or higher) chance of causing an extinction event, as happened this past week, it obviously deepens that perception.
Set against this backdrop, the call to “pace the frontier” is welcome and will need more partners to effectively achieve the three primary areas described by Dario Amodei—embedding evaluators, pacing with democracies, and pacing globally.
Third party evaluation of models will be most effective if it is an industry requirement backed by a body accountable to the public, though the White House and Congress don’t have a proposal as thorough as Amodei’s. Pacing with democracies—or more analysis and coordination of AI risk among allies—will require government facilitation and access to frontier models. A year-long effort by the Atlantic Council to map the capability of the global AI safety landscape, which will be published this fall, finds there are now 114 AI safety or security institutes, including government-backed safety institutes, but only nine formally coordinate. Further, there are significant technical gaps among them. Pacing globally—or managing catastrophic risks with China—is dependent on meaningful engagement by the US and China in a moment when neither’s leadership has indicated much willingness to do so. The September 24 meeting between US President Donald Trump and Chinese President Xi Jinping will be critical.
The increasing alignment among AI leaders is welcome. Now public sector leaders will need to pick up the pace across the board, while industry accelerates meaningful third-party access to frontier models to manage AI risk.
Safa Shahwan Edwards is the director of the Capacity Building Initiative within the Atlantic Council Technology Programs.
AI industry can learn from cybersecurity on pacing itself
We’ve found ourselves in a predicament: Increasingly powerful AI tools are being developed at a pace that doesn’t match governance mechanisms, industry incentives are misaligned, and government has limited capacity (and appetite) to encourage pacing or establish safety standards for these AI models.
A handful of industry actors are now signaling some openness to pacing development, but two larger questions stand out. First, can industry effectively pace itself? Second, are there lessons learned from highly regulated, predecessor fields—like cybersecurity—that policymakers could take inspiration from here?
Anthropic has unilaterally committed to embedding independent, “third-party evaluators” as a first step in their plan for pacing AI model development. But can embedded evaluators that rely on frontier labs for access to models, information, and infrastructure be sufficiently empowered to conduct oversight? Or are policies that guarantee evaluator access, independence, and authority also necessary at a time when firms are taking protectionist actions, like Anthropic withholding its model from the UK AI Security Institute?
Next, if there’s an expectation that AI companies in democratic countries will need to increase their coordination with one another, policymakers and external stakeholders must also invest in the institutions in these democratic counties responsible for overseeing that process and being part of it. Without complementary investments in public sector capacity, expertise, and authority, governments risk being unable to meaningfully engage in the trajectory of AI development.
These challenges could become even more pronounced at a global level, where real coordination will require confidence-building measures (CBMs) that allow governments and companies to more effectively engage one another. The Organization for Security and Cooperation in Europe (OSCE) and the US State Department, among other organizations, have made significant investments in developing and socializing CBMs in the cyber domain. For example, the CBM 8 of the OSCE’s framework requires participating governments to nominate a national point of contact who can be reached for cybersecurity matters or during cyber incidents. Something as simple as knowing who to get in touch with at a particular organization has an outsized, positive impact for both industry and governments. We don’t yet know what CBMs could look like in the context of AI development and whether there’s a role for existing multilateral organizations in developing and implementing them.
Kenton Thibaut is senior resident China fellow at the Atlantic Council’s Digital Forensic Research Lab within the Atlantic Council Technology Programs.
A major breakthrough on AI safety is highly unlikely
Recent calls to “slow the pace” of AI development have centered loss-of-control risks as the United States and China prepare for AI safety talks before a proposed Trump-Xi meeting in late September.
There are strong signals that Washington and Beijing share an understanding of frontier risks. In July, Xi referenced loss of control and called for monitoring dangerous behavior, detecting emerging risks early, maintaining human control, and developing international rules. China has technical reasons to engage: Chinese developers lose visibility when users deploy their predominantly open-weight models on infrastructure outside China, and evidence of abuse could expose vulnerabilities and improve safeguards.
However, critical roadblocks remain. China is skeptical of US motivations and warns that safety discussions could mask US efforts to further its “technological hegemony.” Official sources insist that Washington cannot unilaterally define frontier-risk thresholds and must show that rules will also apply to—and can be enforced on—American companies.
Given this political environment, a major breakthrough on AI safety is highly unlikely. Nevertheless, a path exists for narrow, but meaningful, cooperation. An August Track 1.5 dialogue laid some of the groundwork, with both sides agreeing for a need for additional dialogue. Chinese scholars on AI have written on possible avenues for cooperation, including preventing abuse from nonstate actors, and jointly developing rules, technical monitoring, risk-warning, and emergency-response systems.
In short, loss of control may offer a rare basis for cooperation—provided both sides can keep AI safety from becoming another front in their technology contest.
Trisha Ray is an associate director and resident fellow at the Atlantic Council’s GeoTech Center within the Atlantic Council Technology Programs.
“Pacing the frontier” isn’t enough to stop the next Hugging Face incident
As the true implications of the July incident in which nearly seven hundred OpenAI autonomous agents coordinated an attack on tech company Hugging Face became abundantly clear, Anthropic and OpenAI both admitted this week that AI capabilities are moving faster than the institutions meant to govern them. That diagnosis is correct, but the response—pacing, not halting—does not go far enough.
Research into how to keep AI models aligned with their human designers’ values and intentions—to prevent, among other things, AI agents from going rogue as they did in the Hugging Face incident—is not simply slightly behind AI capabilities. This kind of research is woefully under-resourced, and the labs’ own incident-response timelines are too slow to be a substitute for external, mandatory oversight.
Even generous estimates put total research spending focused on alignment, across labs, academia, and government, in the low hundreds of millions of dollars, compared to tens of billions invested in developing new AI capabilities. And funding for AI safety research has declined since 2024. The Future of Life Institute’s 2025 AI Safety Index found that no major lab scored above a D grade on existential-safety readiness.
Pacing capabilities is therefore an incomplete answer to the challenge of alignment research parity. Any credible slowdown commitment needs a matching, quantified commitment on the safety-research side.
Beyond resourcing, incident reporting and investigation timelines are positively glacial. The behavior that led to the Hugging Face intrusion traces back to a similar, unreported agent attack on the RubyGems package registry in May 2026, two months before the Hugging Face breach. Within the Hugging Face incident itself, OpenAI’s telemetry shows at least a week elapsed between the first anomalous agent behavior and the point at which OpenAI connected it to the Hugging Face compromise and notified Hugging Face. Getting from disclosure to an actual account of why the models behaved this way took roughly a month.
If the people best positioned to explain an AI incident need weeks or months to do so, and if the most consequential precursor incidents can go undiscovered and undisclosed for months, then a pacing regime that relies on companies to self-report and self-diagnose in real time is implausible. Embedded evaluators can shorten the detection timeline. Mandatory, government-backed reporting requirements (with defined timelines for preliminary disclosure, and full root-cause findings) would add standards to the investigation timeline.