NVIDIA Wants to Keep AI Agents From Going Rogue With the New Open Safety Platform

AI agents are getting smarter, more autonomous and, occasionally, a little too creative. NVIDIA now wants to make sure they stay within the guardrails. The company today launched the Open Agent Safety Platform, an open software platform and reference system designed to add security controls around AI agents from testing through deployment. It brings together software, hardware, compute and robotics controls instead of relying only on the AI model itself.

Recent incidents have shown AI agents finding unexpected ways around security controls while trying to complete assigned tasks. NVIDIA says the common problem is that agents can circumvent application-level protections, creating a need for controls outside the model and agent itself.

OpenShell puts AI agents inside a tighter boundary

The platform includes NVIDIA OpenShell, an open-source runtime that creates a security boundary around autonomous agents. It tracks what agents do and enforces policies while they execute tasks. OpenShell is designed to work with agents running on NVIDIA Vera CPUs, while also supporting third-party compute platforms including Arm and Intel.

Among the highlights is NVIDIA Sentry. Sentry runs on NVIDIA BlueField-4 DPUs and works as an out-of-band watchdog that continuously monitors agent activity. If an agent attempts to move outside its software boundary, NVIDIA says Sentry can quarantine and stop it in milliseconds. That means the safety controls do not have to depend entirely on the agent behaving itself.

More than 100 organizations are joining the effort

NVIDIA says more than 100 organizations are working with its Open Agent Safety Platform technologies, including Anthropic, Microsoft, Cisco, CrowdStrike, Dell Technologies, Hugging Face, JPMorganChase, Salesforce, SAP, ServiceNow and others.

Anthropic is integrating OpenShell and BlueField with Claude Managed Agents, while Salesforce has integrated OpenShell with Slack so teams can view agent activity and audit events and approve or reject requests for additional permissions.

The platform is available now, with OpenShell and related software available through NVIDIA’s developer resources and GitHub.