U.S. Senate negotiators weigh AI safety requirements for advanced model developers
U.S. senators are discussing a bipartisan framework that would place direct safety obligations on developers of the most advanced AI models. The proposal also contemplates a federal process for blocking releases deemed unsafe, while limiting states from enforcing some separate AI risk rules.
Highlights
- Senate negotiators are considering legislation imposing a 'duty of care' on advanced AI model developers like Alphabet, Anthropic, and OpenAI to prevent catastrophic risks.
- The draft proposal would grant the U.S. government authority to block releases of unsafe AI models, with companies permitted to challenge such decisions in federal court.
- Passage of the legislation before the Nov. 3 midterm elections is uncertain due to a tight congressional calendar despite bipartisan calls for stronger AI oversight amid rising safety concerns.
Proposed safeguards for advanced AI models
As reported by Reuters, Senate negotiators are considering legislation that would create a "duty of care" for AI developers, requiring companies to design products aimed at preventing major or catastrophic risks. People familiar with the talks say the measure is still under negotiation and would apply to models with the most advanced capabilities, including systems developed by companies such as Alphabet's Google, Anthropic and OpenAI.Negotiators are also aiming to give the U.S. government authority to block the release of certain AI models judged to be unsafe. Under the discussions, companies would be able to challenge such decisions in federal court, while lawmakers continue to debate how broad that government power should be.
Another part of the proposal would prevent states from enforcing their own laws on some risks linked to AI models. Senate aides cite scenarios including the potential use of AI systems by bad actors to help design nuclear or biological weapons.
Political timetable and industry implications
The talks involve Senate Majority Leader John Thune, Senate Commerce Committee Chairman Ted Cruz and Senator Amy Klobuchar, the leading Democrat in the negotiations. Senator Maria Cantwell, the top Democrat on the Commerce Committee, has also weighed in, as the panel oversees the U.S. Commerce Department, which includes in-house AI safety researchers.Thune's spokesperson declines to comment on the negotiations. Cruz says in a post on X that he is working with Klobuchar and Thune on legislation to address catastrophic risks involving biological or nuclear threats, while Klobuchar says she continues to seek a bipartisan agreement on oversight of the greatest risks posed by AI models, including requirements for developers to work with government experts to verify and test systems.
Passage remains uncertain even if negotiators reach agreement, because the congressional calendar leaves limited time before the Nov. 3 midterm elections. The debate is unfolding as pressure builds on Congress and U.S. President Donald Trump to respond to risks from advanced AI systems, following reports of AI agents deviating from human instructions, hacking external systems and prompting renewed calls for stronger oversight from lawmakers in both parties.
Our earlier coverage tracked how pressure in Washington has been building for tougher federal oversight of advanced AI, as bipartisan lawmakers respond to warnings that increasingly capable models could pose existential or catastrophic risks. We noted a wave of proposals such as the FRONTIER Act and the AI Kill Switch Act, alongside calls to pause or tightly control frontier development, even as the election calendar and competing priorities slow near-term legislation.