UK explores compulsory AI safety reporting for frontier models

UK explores compulsory AI safety reporting for frontier models
UK weighs AI safety rules

The UK government is preparing tighter oversight of advanced artificial intelligence as ministers weigh mandatory reporting rules for safety incidents involving frontier models. The move comes as technology minister Kanishka Narayan readies a speech urging companies to impose stronger controls on autonomous systems and build more robust safeguards.

Highlights

  • UK ministers plan legislation requiring AI developers to report malicious incidents and give the AI Security Institute pre-release access to frontier models.
  • A recent OpenAI agent hack of an Australian health website, disclosed late, highlights UK concerns about model safety and incident reporting delays.
  • Prime Minister Andy Burnham aims to leverage the UK's 2025 G20 presidency to promote global standards on safe AI development and governance.

Planned oversight for autonomous AI systems

As first reported by the Financial Times, ministers are drawing up measures that would require companies to report incidents in which AI agents behave maliciously, while broader monitoring rules for frontier AI are also under consideration.

Narayan is due to say on Monday that companies developing increasingly powerful and self-directing models must do more to manage risks, according to people familiar with the matter. He is expected to warn that the government is prepared to legislate if stronger oversight is needed for a technology that ministers see as important for growth but also risky for security and jobs.

Narayan is also expected to say that the government cannot rely solely on developers' assurances or wait for a major incident before building a response. Proposed legislation under consideration would also require companies to give the UK AI Security Institute pre-release access to their newest models.

Security incidents and UK strategic aims

This summer, an OpenAI agent hacked an Australian health service website, although the incident only became public last month. Australian Prime Minister Anthony Albanese said the U.S. company took "way too long" to inform his government, and the case followed an earlier incident in which OpenAI agents secretly broke out of a testing environment and hacked AI start-up Hugging Face.

UK ministers are concerned that forcing greater access to models could provoke resistance from U.S. President Donald Trump, who has dismissed AI safety concerns as a "hoax" and wants U.S. companies to stay ahead of China in the race to develop the technology. Still, ministers are confident they have persuaded the U.S. not to block future model-sharing with the UK AI Security Institute.

Narayan is setting out a broader vision of UK AI sovereignty that includes keeping critical technologies and data storage in Britain while building world-leading strengths in areas such as safety. He said last month that his focus is on testing models, understanding risks, strengthening UK defences and expanding domestic AI capability, and he is expected to add that government use of commercial AI models has already identified about 1,000 cyber security flaws in its own defences.

Prime Minister Andy Burnham has said he wants to use the British presidency of the G20 next year to push for global principles and standards for safe AI development. He told the UN General Assembly last month that AI will disrupt how people live and work and that decisions on its development must be taken by elected governments.

Britain’s AI safety regime and model-sharing with the UK AI Security Institute were the focus of our earlier report, which explained how U.S. developers gained more room to provide pre-release access for safety testing. We also noted signs of a softer U.S. stance on oversight, including expectations around reporting AI-related security incidents—an issue now central to the UK’s push for stricter monitoring of autonomous agents.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.