Nvidia Announces AI Agent Safety Platform

Big AI claims that its AI agents are so powerful that they’re “going rogue.” But Nvidia, keen to keep the AI gravy train rolling, says it has a solution, which it calls the Open Agent Safety Platform.

“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Nvidia founder and CEO Jensen Huang says. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering. The Nvidia Open Agent Safety Platform brings together industry, researchers, and public-sector organizations to share best practices, align on evaluation methods and foster international cooperation. Together, we can raise the bar for global AI safety.”

Nvidia describes its Open Agent Safety Platform as an open software platform and reference system design to strengthen AI security from agent testing to deployment. It’s a response to recent reports of AI agents from OpenAI, Anthropic, and other companies bypassing security systems and provides governance and control across software and the hardware, compute, and robotics systems that run these AI agents. The Open Agent Safety Platform includes the following components:

• Nvidia OpenShell. This secure runtime software controls how autonomous AI agents execute tasks across open and closed models on CPUs through sandboxing and policy.
• Nvidia Vera. This is Nvidia’s first CPU that was custom-built for agentic AI, though Nvidia OpenShell is open source and can thus be customized to run on other compute architectures, like those from Arm and Intel.
• Nvidia Sentry. This optional security layer is built on Nvidia DOCA and runs on BlueField-4 DPUs alongside OpenShell. It continuously monitors AI agent security and integrity by looking for deviations from their designed intent based on predefined behavioral profiles.
• Claude Managed Agents. This optional security layer was created in partnership with Anthropic. It provides a security boundary by running the agent loop in a separate server from the sandboxes where their work executes.

“Companies are giving AI agents more of their most important work, and they need to direct and verify what those agents do, especially in sensitive environments,” Anthropic chief commercial officer Paul Smith says. “Claude Managed Agents gives companies a clear view of what each agent is doing, and Nvidia’s platform adds another layer of governance and control across hardware and software.”

The Nvidia Open Agent Safety Platform software is supported by a laundry list of Big Tech, Big AI, and other companies that includes Anthropic, Cisco, CrowdStrike, Dell, Figure, HPE, Hugging Face, JPMorganChase, Microsoft, Palantir, Palo Alto Networks, Perplexity, Red Hat, Salesforce, SAP, Scale AI, ServiceNow, and SpaceX AI. It’s available now on Nvidia developer resources and GitHub.