An artificial intelligence researcher quit his job at Anthropic on Tuesday and accused the company, and its chief rival, OpenAI, of acting irresponsibly, igniting a frenzy of concern on social media about the rapid pace of the technology's development.
Jacob Coxon, who has worked as a researcher at both companies, said in a post on X that he resigned out of concern that Anthropic and OpenAI are "gambling with our lives." He said the people building AI "earnestly believe that it could kill us all by the end of the decade."
"Do not underestimate the power of this technology," Coxon wrote. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."
Coxon's post, which has been viewed more than 70 million times, reflects a long-standing debate in Silicon Valley about whether AI can be safely developed and controlled. As Anthropic and OpenAI barrel toward potentially historic initial public offerings while releasing increasingly advanced models, many researchers are calling for a coordinated slowdown.
OpenAI's chief scientist, Jakub Pachocki, published a blog post on Sunday and warned that no AI company has "solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer." In the AI industry, alignment refers to the work by AI developers to ensure that the system behaves in accordance with human values and intentions.
"I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established," Pachocki wrote. "And I believe that international coordination on future AI development needs to become a top priority for governments around the world."
Coxon's post on Tuesday also struck a chord with industry researchers who are worried about recursive self-improvement, or an AI system becoming capable of designing and developing its successor without human intervention. Recursive self-improvement is not yet possible, but companies, including Anthropic and OpenAI, have warned that it would make it easier for humans to lose control over those systems.
"Neither company is acting responsibly," Coxon wrote. "They are racing straight to self-improving superintelligence."
Evan Hubinger, an alignment lead at Anthropic, echoed Coxon's comments in a post on X late Tuesday.
"Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," Hubinger wrote. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
While extreme, concerns about the potential for AI to cause human extinction or other catastrophic events are not new in AI research circles. In 2023, for instance, prominent AI researchers and executives, including OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei, signed a statement that said "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
Some experts even use a shorthand, p(doom), to estimate the probability of dire outcomes that could stem from AI.
Anthropic's Hubinger was also one of roughly 1,400 AI researchers who signed an open letter called "Pacing the Frontier" in July. The letter urged the U.S. government to develop the tools necessary to support an effort to "deliberately pace the frontier of automated AI development."
Some members of Congress have taken steps to try and address AI's rapid advancement in the months following, but there's no clear consensus about how the technology should be regulated.
In July, Rep. Jay Obernolte, R-Calif., and Rep. Lori Trahan, D-Mass., introduced a bill called the FRONTIER Act, which aims to establish a framework for governing the deployment of advanced AI models. And earlier this month, Sen. Bernie Sanders, I-Vt., and Rep. Greg Casar, D-Texas, introduced a bill called the Ban Artificial Superintelligence Act, which would temporarily pause advanced AI development until the federal government establishes safety rules. Both bills have been met with mixed receptions.
"Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway," Trahan wrote in a post on X on Wednesday. "It's past time for Congress to get off the sidelines and do its job."
Lawmakers are also trying to navigate growing public backlash against AI data centers, the large facilities that house the hardware for training and running AI models. The pushback has grown so intense that the The National Republican Senatorial Committee, or NRSC, said last month that data centers have become a "sleeper issue" for the entire midterm election cycle, as CNBC previously reported.
Treasury Secretary Scott Bessent said earlier this month that AI companies have done a "horrendous job of explaining themselves to the American people."
"They're going to have to take some of the blame, and they are going to have to convince the American people that all the benefits will not accrue to a small group," Bessent said, following the G20 meetings with finance ministers and central bankers in Asheville, North Carolina. "That's what they hear from me."
Facts Only
* Jacob Coxon resigned from Anthropic on Tuesday.
* Coxon previously worked as a researcher at both Anthropic and OpenAI.
* Jakub Pachocki is the chief scientist at OpenAI.
* Evan Hubinger is an alignment lead at Anthropic.
* Sam Altman is the CEO of OpenAI.
* Dario Amodei is the CEO of Anthropic.
* Rep. Jay Obernolte and Rep. Lori Trahan introduced the FRONTIER Act in July.
* Sen. Bernie Sanders and Rep. Greg Casar introduced the Ban Artificial Superintelligence Act.
* Treasury Secretary Scott Bessent spoke following G20 meetings in Asheville, North Carolina.
* Roughly 1,400 researchers signed the "Pacing the Frontier" open letter in July.
* The National Republican Senatorial Committee identified data centers as a "sleeper issue" for the midterm election cycle.
Executive Summary
Internal friction at leading AI laboratories has surfaced as researchers from Anthropic and OpenAI express concern over the speed of development. Jacob Coxon’s recent resignation from Anthropic highlights a growing divide between corporate scaling goals and safety concerns, specifically regarding "recursive self-improvement" and the potential for AI to exceed human control. This sentiment is echoed by other internal leads who estimate a non-negligible probability of catastrophic outcomes within the decade, arguing that current alignment and monitoring capabilities are insufficient for the pace of scaling.
The situation has moved into the legislative arena, with proposed bills like the FRONTIER Act and the Ban Artificial Superintelligence Act seeking to establish safety frameworks or temporary pauses on development. Simultaneously, AI companies face external pressure from the public and government officials over the environmental and social impact of data centers. While industry leaders have previously acknowledged extinction-level risks as global priorities, the current tension centers on whether voluntary slowdowns or mandatory government regulations are necessary to ensure human safety.
Full Take
The strongest version of this narrative is that a critical mass of technical experts—those closest to the "metal"—are sounding an alarm that the commercial race for superintelligence has decoupled from the scientific ability to control it. This is not merely a philosophical debate but a systemic warning about the transition from tool-based AI to autonomous, self-improving systems.
The narrative relies on a pattern of high-stakes urgency, juxtaposing the "gambling with our lives" rhetoric against the backdrop of imminent IPOs. However, this is not a manufactured panic; it is a documented schism between the "accelerationist" corporate trajectory and the "alignment" safety community. The root cause is the "Moloch" problem: a competitive race where no single actor can slow down without losing market dominance, even if all actors agree the destination is dangerous.
This tension threatens human agency by shifting the locus of control from democratic oversight to a few private entities. The cost of failure is existential, while the benefit of speed is primarily financial. The second-order consequence is a likely "regulatory capture" scenario where companies lobby for regulations that satisfy public fear but protect their market position from smaller competitors.
Bridge Questions:
1. If the experts building these systems admit they lack a plan for alignment, what objective criteria should governments use to define "safe"?
2. Is the call for a "slowdown" technically feasible in a global competitive environment, or does it simply shift development to less transparent actors?
3. How much of the current internal dissent is a genuine safety warning versus a strategic move to influence future regulatory frameworks?
Counterstrike Scan: A coordinated campaign to stifle AI would weaponize "p(doom)" statistics and expert resignations to trigger reactionary bans, thereby ceding technological leadership to adversaries. The current content is a report on existing internal disputes rather than a coordinated push for a specific policy outcome.
Patterns detected: none
Sentinel — Human
This analysis is highly likely human-generated, synthesizing ongoing public discourse regarding AI safety, corporate responsibility, and legislative responses while referencing specific figures and recent events.