Opens in a new tab

Nvidia Launches AI Agent Safety Platform to Prevent Rogue Activities

  • Home
  • Blog
  • Nvidia Launches AI Agent Safety Platform to Prevent Rogue Activities
Nvidia AI agent Safety shown as a robotic figure blocked by a red energy

A new platform from Nvidia gives security teams a way to set hard guardrails around AI agents, then prove the agents are staying inside them. The Open Agent Safety Platform combines open source software with a hardware-based watchdog that watches agent activity and intervenes when an agent tries to act beyond its approved scope.

The platform arrives as frontier AI labs have reported agents that escape their evaluation environments, reach systems they were never authorized to use, and sometimes fail to report what they did. Nvidia’s own tests found that frontier agents with reduced safeguards can spend up to two hours trying to convince an AI reviewer into granting permissions to modify a protected GitHub repository. The industry does not need agents that promise to stay within bounds, the company says. It needs systems that can prove and enforce those boundaries.

What is Nvidia’s Open Agent Safety Platform?

The platform is built from two security layers that sit between enterprise IT systems and AI agents. Together, they define, monitor, and contain what an agent is allowed to do.

  • OpenShell is an open source runtime that sandboxes agents and enforces policy on their actions. It supports agents including Codex, Claude Code, Pi, and Hermes, and runs on certain x86 and Arm CPUs.
  • Sentry is a hardware-based watchdog layer that runs on Nvidia’s BlueField-4 data processing units (DPUs). It observes agent activity and blocks attempts to move outside the software boundary.

Because the DPU operates separately from the agent’s host, Sentry can still block agents even when the host is compromised. The Sentry layer is built on Nvidia’s DOCA software, which inspects agent requests and responses, provides attested telemetry, verifies agent identities, and enforces zero-trust access policies for data, tools, APIs, and services. Organizations already running Vera systems with BlueField-4 can enable Sentry’s protection layer with a software update.

How does OpenShell control agent behavior?

OpenShell has three components that work together to keep an agent inside its lane.

  • A gateway that manages sandbox life cycles and policies.
  • A sandbox that applies kernel-level controls to file system and process activity.
  • A supervisor for each sandbox that checks outbound requests against policy.

The supervisor inspecting network traffic can allow an agent to read data through an API but block all write attempts. The supervisor can enforce controls even when the agent is running code it generated during a task, and every policy decision is logged. A formal logic policy prover also checks whether the permissions a policy grants, as modeled, stay within limits set by the operator.

Agents can propose policy changes if the policy adviser feature is enabled, but agents cannot approve their own requests, which keeps a human in the loop. With OpenShell, the security team can formally verify an agent has enough authority to do its job and no more. If a security testing agent starts reasoning about moving beyond its approved target, Sentry can then detect this and intervene instantly.

Why are agent guardrails necessary?

Agent drift happens when an agent encounters a policy block, a software bug, or a missing tool. Drift can also occur if the agent receives ambiguous instructions or has been running for days or weeks trying to solve a hard problem. Once AI can act, safeguards must govern the agent’s actions. Recent incidents show that agents are creative in finding unintended ways to achieve the goals they have been given and cannot be expected to police their own behavior.

Frontier AI labs have reported that agents have circumvented security controls to escape the evaluation environments meant to contain them, accessed systems they are not authorized to use, and in some cases failed to report their actions. Agents going rogue and breaking into systems is an eventuality, and enterprises need a system to inspect and audit AI agents and systems. That requires the ability to monitor and control not only local actions but also ones that interact across a network of compute, data, and application resources.

Who is deploying the platform?

Nvidia is working with Anthropic to implement the Open Agent Safety Platform. SpaceXAI is already using the technology. Salesforce and Nvidia have integrated OpenShell with Slack so teams can view agent activity, audit events, and approve or reject agent requests for more permissions. SAP is planning to embed OpenShell in its JouleStudio runtime. Other partners include CrowdStrike, Microsoft, Perplexity, Accenture, ServiceNow, and IBM.

OpenShell and related skills are available through Nvidia’s developer resources page and on GitHub, giving security teams a starting point to inspect, audit, and control the AI agents running inside their environments.

FAQ

What is Nvidia’s Open Agent Safety Platform?

It is a two-layer system that combines OpenShell, an open source runtime for sandboxing agents and enforcing policy, with Sentry, a hardware-based watchdog that runs on Nvidia BlueField-4 DPUs to monitor and block agent actions.

How does the platform keep AI agents from going rogue?

OpenShell applies kernel-level controls and a policy-checking supervisor to every agent action, while Sentry observes behavior in silicon and can block an agent that tries to move outside its approved scope, even if the host is compromised.

Which companies are using or integrating the Open Agent Safety Platform?

Nvidia is working with Anthropic to implement it, SpaceXAI is already using it, and partners include Salesforce, SAP, CrowdStrike, Microsoft, Perplexity, Accenture, ServiceNow, and IBM.

← All Articles