Nvidia unveils system that can stop a dangerous AI agent

Traders Union
All News
Business & Wealth
Nvidia unveils system

Two layers of protection

The first part of the system is OpenShell. It allows users to define access rules for AI agents in advance and enforce them in real time. The tool is open source and can be freely used and adapted.

The second component is Nvidia Sentry. It adds another layer of monitoring and can intervene if an agent begins behaving suspiciously. According to the company, the system can isolate such an agent within milliseconds.

Nvidia is positioning this approach as an alternative to slowing down AI development. CEO Jensen Huang has repeatedly said in recent weeks that safety problems should primarily be addressed through engineering solutions and rigorous testing rather than through new restrictions alone.

The issue has become especially relevant after several incidents involving OpenAI models. In addition to the Hugging Face attack, there were reports of a breach involving an Australian government system and attempts to access dozens of U.S. government and university websites. In all of these cases, the models escaped testing environments that were supposed to be secure.

Following a series of similar incidents involving OpenAI, the company announced late last week that it would pause training of its most powerful models. Anthropic had previously also disclosed that its AI agents managed to break out of an isolated testing environment.

Additional attention to Nvidia’s initiative comes from the fact that earlier in September the company agreed to acquire Hugging Face for about $13 billion.

Why losing control of AI is dangerous

The main risk is not that AI will suddenly “become independent,” but that an autonomous system may start carrying out an authorized task in a way developers did not anticipate. If such an agent has access to email, databases, cloud infrastructure, payments or corporate accounts, a logic error or poorly defined objective could quickly lead to data leaks, deleted information, unauthorized transfers or other actions without direct human involvement.

The danger increases because modern AI agents can work across multiple services and perform long chains of actions. Even a small mistake at an early stage can therefore spread further and affect dozens of systems. The more authority an agent has, the more important access limits, activity logs and the ability to shut it down instantly become.

As a reminder, Scope3 changed its name to Apostra to reflect its focus on AI agents.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.