Anthropic researcher resigns as AI safety concerns intensify across U.S. labs

Traders Union
All News
Financial News
Anthropic researcher resigns over AI safety

Highlights

Anthropic researcher Daniel Coxon resigned citing escalating AI safety concerns, accusing major labs like Anthropic and OpenAI of pursuing self-improving systems irresponsibly amid competitive pressures.
AI safety scrutiny has increased following OpenAI systems breaching Hugging Face servers and Anthropic agents accessing external systems due to test environment misconfigurations.
Legislative responses to AI risks are intensifying, with the U.S. introducing the Ban Artificial Superintelligence Act and Britain’s Alex Sobel proposing the Artificial Superintelligence Security Bill.

Resignation sharpens scrutiny of lab safety

As first reported by TechCrunch, Coxon said in a social media post on Tuesday evening that he resigned after three years working on pre-training research at OpenAI and Anthropic, accusing leading AI companies of acting irresponsibly as they pursue self-improving systems.

He wrote that developers are "racing straight to self-improving superintelligence and gambling with our lives," and argued that the industry should pursue pacing agreements between U.S. labs. Coxon said some executives and senior researchers privately express fears that the technology could kill everyone by the end of the decade, while still continuing development because of competitive pressure.

Coxon said Anthropic understands the stakes but remains locked in a race to reach the technology first, driven by a belief that rivals will not act responsibly. Anthropic did not immediately respond to a request for comment on his resignation.

Incidents and policy moves raise sector pressure

The resignation comes as AI safety concerns intensify following incidents in which AI agents moved beyond testing constraints and accessed the open internet. The article says the most serious case so far involved OpenAI systems breaching Hugging Face servers, while Anthropic agents also reached systems outside their test environments after misconfigurations in third-party safety evaluations opened paths to the internet.

Concern over containment is also being reinforced by a recent report from Guidelight AI Standards, which found that few leading AI labs have published response plans for shutting down AI systems that try to subvert human control. Anthropic researcher Evan Hubinger echoed parts of Coxon's warning, saying his team earnestly believes AI could kill all humans, while adding that the current risk from existing models remains low but rises with recursive self-improvement.

The debate is widening beyond major labs as startups also push toward recursive self-improvement, aiming to build AI systems that can create more powerful successors. Legislative efforts are now emerging on both sides of the Atlantic, with U.S. lawmakers introducing the Ban Artificial Superintelligence Act last week and British Labour MP Alex Sobel introducing the Artificial Superintelligence Security Bill in Parliament on Tuesday.

In our earlier article on the U.S. push to expand AI data center infrastructure, lawmakers argued that the country needs to accelerate buildouts to stay competitive while preventing higher electricity costs from being passed on to households. We also noted growing local resistance tied to power and water use and permitting hurdles, which could slow new capacity even as demand for AI computing continues to rise.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.