NeMo Agent Toolkit Security: NVIDIA Agent Toolkit Authentication, User Identity and Tool Controls After v1.9.0
NVIDIA NeMo Agent Toolkit 1.9.0, published on September 10, 2026, carries exactly three breaking changes, and all three move the agent control plane: the local code sandbox is gone, memory tools no longer accept a user ID chosen by the model, and identity resolution got stricter. If you run the toolkit in production, the upgrade is also a security review.
Direct answer
NeMo Agent Toolkit is NVIDIA's open-source framework for building and profiling agent workflows across LangChain, LlamaIndex, CrewAI and plain Python. Its security model is explicit about who owns what: the toolkit resolves a user identity from each request, but NVIDIA's own documentation states that identity resolution "is an identity mapping step, not an authentication or authorization layer," and that without upstream verification "any party that can send HTTP requests to NeMo Agent Toolkit can impersonate any user." Version 1.9.0 adds opt-in human-in-the-loop middleware, a fail-closed option for output verification and a circuit breaker. What it does not provide is a policy decision on each tool call. AgentShield supplies that layer: per-agent tool and data permissions, approval gates on irreversible actions, injection screening of tool output, and an audit record kept outside the workflow.
Try it live
Watch AgentShield block an attack in real time.
Pick a scenario and drive the inspection lane yourself. No signup needed.
Run a request
Runs the live engine on your text. Nothing is stored, no account needed.
Inspection lane
INSPECTINGPolicy trace
High-risk action held for approval
Audit trail
- § · → → →
The risk
A platform team runs a NeMo Agent Toolkit workflow behind an internal chat UI. It has a retriever over the support knowledge base, a code execution tool for ad hoc analysis, a memory store for per-user context and an MCP connection to the ticketing system with write scope. The September upgrade to 1.9.0 breaks the build twice: the local sandbox they used for code execution no longer exists, and the memory tools stop accepting the user ID the model used to pass in. Fixing the build is a day of work. The harder question is the one the release notes raise without answering: if the model could choose whose memory to read until last week, what else in this workflow is trusting the model to make an access decision?
How AgentShield handles it
Keep the toolkit for what it is good at, orchestration, profiling and evaluation, and move the access decisions out of it. Validate identity before a request reaches the server. Give each workflow an explicit allowlist of tools and resources, so a manipulated model cannot reach a tool it was never granted. Route irreversible actions, such as ticket closure, outbound email or writes to production data, through an approval gate. Inspect retriever and tool output for injected instructions before the next step acts on it, and write every decision to a record the workflow itself cannot edit.
The controls
The controls that secure what a NeMo Agent Toolkit workflow may call, for whom, and with whose approval.
What NeMo Agent Toolkit 1.9.0 changed, and why each breaking change is a security change
Most framework releases break things for ergonomic reasons: a renamed parameter, a moved import. The 1.9.0 release notes list three breaking changes and every one of them narrows what an agent can do or whom it can act as. That is a useful signal about where NVIDIA saw risk, and a useful checklist for your own deployment.
| Breaking change | What it does | The security reading |
|---|---|---|
Remove local_sandbox (PR #2194) | Python code execution now runs against a remote sandbox server, with Piston as the documented default | Model-written code no longer runs next to the workflow process by default. The sandbox is now a separate service you have to deploy, secure and network correctly |
Do not expose the user_id parameter to the LLM (PR #2190) | Memory tools take the user from configuration or the authenticated request, not from an argument the model fills in | Before this, a prompt injection could in principle steer the model to read or write another user's memory. The model is no longer the party deciding whose data it touches |
| Improved user identity resolution (PR #2197) | Identity is resolved from the request credential, with header-based identity made opt-in | Fewer implicit ways for a request to claim an identity. It is still identity mapping, not authentication, which is the next section |
The same release adds three controls you have to switch on: HITLMiddleware for human-in-the-loop function interception (PR #2060), a fail_closed boolean on output_verifier_tools (PR #2187), and CircuitBreakerMiddleware for tool fault tolerance (PR #2158). They are welcome. They are also opt-in, and the middleware documentation lists every auto-discovery flag, from register_llms to register_workflow_functions, as defaulting to False. A middleware you configured for one tool does nothing for the tool someone adds next month.
One of the three, the circuit breaker, is easy to misread as a security control. It is a reliability pattern: after a set number of consecutive failures it stops calling a failing function for a cooldown period. It will not stop a tool call that succeeds and should not have happened, which is the case that matters when an agent has been manipulated.
Is NeMo Agent Toolkit authentication enough for production?
No, and NVIDIA says so directly. The toolkit inspects each incoming request for a session cookie, a JWT bearer token, an API key or HTTP Basic credentials, and turns whatever it finds into a stable user_id. That ID then controls access to per-user workflow state, including conversation history and cached tokens. The documentation for 1.9.0 is candid that JSON Web Tokens are decoded without verifying the signature, so the server accepts whatever credential it is handed. The same warning continues: "In production, deploy an authenticating reverse proxy or auth middleware that validates JSON Web Tokens before they reach NeMo Agent Toolkit. Without upstream verification, any party that can send HTTP requests to NeMo Agent Toolkit can impersonate any user."
That is a clear division of labor rather than a flaw, and the fix for the authentication half is something you almost certainly already own: an API gateway, an identity-aware proxy or your ingress controller validating tokens against your identity provider. You do not need to buy anything for that part, and we would not sell it to you.
The part that is left over is authorization, and it is the part that decides the size of an incident. Once a request is authenticated, which tools may this workflow call on that user's behalf, against which resources, and which of those calls need a person to approve them? The toolkit supports OAuth2 scopes on MCP service accounts and the auth providers you configure, which governs what a credential can reach. It does not decide whether this particular call, with these arguments, produced by a model that just read a customer email, should run.
| Question | Who answers it | What to use |
|---|---|---|
| Is this request really from the user it claims to be? | You, upstream of the toolkit | Your gateway or identity-aware proxy validating the JWT signature |
| Which user's memory and history does this request see? | The toolkit, from the resolved identity | Built in, and safer since 1.9.0 because the model no longer supplies the user ID |
| What can the credentials behind each tool reach? | You, at the identity provider | OAuth2 scopes on MCP service accounts, least-privilege database roles |
| Should this specific tool call run right now? | Nobody, by default | A per-agent tool permission policy evaluated on every call, with human approval gates for irreversible actions |
| What exactly did the agent do, and was it allowed? | Partly the toolkit's telemetry | An append-only audit trail that records the verdict, kept outside the workflow |
The risks NVIDIA lists, and which ones the toolkit leaves to you
NVIDIA's security considerations page for the toolkit is worth reading in full because it is unusually specific. It opens its tool-abuse section with a sentence every agent team should keep in front of them: "Large Language Models (LLMs) can make mistakes or can be manipulated to take unintended actions." It then lists what those mistakes look like when an agent holds powerful tools: unauthorized data access, data modification or deletion, unintended API calls, command execution, and resource exhaustion.
The recommended approaches that follow are categories, not features: guardrails for input and output validation, role-based access control to limit agent permissions to specific resources, rate limiting, and sandboxing. Some of those map to things the toolkit ships. Most map to things you build or buy. Reading the two lists side by side is the fastest way to scope a production review.
| Risk named by NVIDIA | What the toolkit gives you | What is still yours |
|---|---|---|
| Unauthorized data access | Per-user workflow state keyed to the resolved identity; memory tools scoped since 1.9.0 | Scoping retrievers and database tools to the data each workflow needs, enforced outside the prompt |
| Data modification or deletion | HITL middleware you can attach to specific functions | Deciding which actions are irreversible and making approval mandatory for them, not optional per function |
| Unintended API calls | MCP client with OAuth2 and service account auth | An allowlist of which MCP tools and arguments each agent may use, checked on every call |
| Command execution | Remote code execution sandbox, local sandbox removed in 1.9.0 | Hardening and isolating the sandbox server, which defaults to a plain HTTP address on localhost in the documentation |
| Resource exhaustion | Timeout and circuit breaker middleware | Budget and rate limits per agent and per user |
The logging section of the same page deserves equal attention. The toolkit's observability features can capture full prompts and responses, and NVIDIA flags credential logging, sensitive data in logs and log access control as risks in their own right. If your traces flow to a shared observability backend, the people who can read traces can read whatever the agent read. Decide that deliberately rather than inheriting it from a default exporter.
There is also a patch history to account for. NVIDIA's November 2025 security bulletin covered CVE-2025-33203, a server-side request forgery in the chat API endpoint of the NeMo Agent Toolkit UI, rated 7.6, affecting UI versions before 1.3.0. If you deployed the reference UI early and have not updated the submodule since, check it.
Where AgentShield fits in a NeMo Agent Toolkit deployment, and where it does not
We would rather you spend nothing than spend on the wrong layer, so here is the split we give teams who ask.
| What you need | Do you need AgentShield? | The straight answer |
|---|---|---|
| JWT validation in front of the toolkit server | No | Your existing gateway or identity-aware proxy does this. NVIDIA recommends exactly that |
| Topic control and conversational rails in a chat assistant | No | NeMo Guardrails is NVIDIA's own open-source answer and it is good at dialogue rails. See our open-source guardrails comparison |
| A single approval step on one sensitive function | Probably not | The new HITLMiddleware covers it if one team owns the workflow and the function list rarely changes |
| One policy for which tools every workflow may call, enforced on every call | Yes | Allowlists and argument scopes evaluated at the action boundary, outside model-influenced code |
| Screening retriever and tool output for injected instructions before the next step acts | Yes | The prompt injection firewall inspects untrusted content on the action path |
| Consistent controls across NeMo, LangGraph and hosted agents | Yes | The toolkit wraps other frameworks, and most estates end up mixed. Policy should follow the action, not the framework |
| Evidence for an auditor of what each agent did and why it was allowed | Yes | An append-only record with the policy verdict on every row, exportable to your SIEM |
The toolkit's own architecture makes the last two rows more pressing than they look. NeMo Agent Toolkit is designed to sit on top of other frameworks, which is its strength: a team can profile a LangChain agent and a CrewAI crew in one place. It also means the security properties of any given workflow depend on the framework underneath it. Our LangChain security page covers the most common one, and MCP server security covers the tool connections most NeMo workflows end up using.
Two honest limits. We do not replace the toolkit, its profiler or its evaluation harness, and we do not make an unauthenticated server safe to expose. Put the proxy in front first. What we add is the decision on each action, which is the one piece of the stack that no framework, NVIDIA's included, currently makes for you.
FAQ
Common questions about nemo agent toolkit security.
Is NeMo Agent Toolkit secure?
It is a well-documented framework with a clear shared-responsibility model, and version 1.9.0 tightened sandboxing, memory scoping and identity resolution. It is not a complete security layer by design. NVIDIA states that identity resolution is not authentication or authorization, so production deployments need upstream JWT validation plus your own tool authorization, approval gates and audit.
Does NeMo Agent Toolkit have authentication?
It has authentication providers for outbound connections, such as OAuth2 for MCP servers and service accounts, and it resolves a user identity from incoming requests. That inbound resolution does not verify JWT signatures, so NVIDIA recommends an authenticating reverse proxy or middleware that validates tokens before requests reach the toolkit.
What changed in NeMo Agent Toolkit 1.9.0?
Version 1.9.0, released September 10, 2026, removed the local code sandbox in favor of a remote sandbox server, stopped exposing the user_id parameter of memory tools to the model, and improved user identity resolution. It also added HITL middleware, a fail_closed option on output verifier tools and a circuit breaker middleware.
Does NeMo Agent Toolkit support human in the loop?
Yes. Version 1.9.0 adds HITLMiddleware, which intercepts function calls so a person can approve them before they run. It is opt-in and attached per function or component. For a single sensitive function in one workflow that is often enough. Across many workflows, a central approval policy keyed to the action type is easier to keep complete.
What is the difference between NeMo Agent Toolkit and NeMo Guardrails?
NeMo Agent Toolkit builds, connects and profiles agent workflows across frameworks. NeMo Guardrails adds programmable rails to LLM conversations, such as topic control, jailbreak screening and output moderation. They complement each other. Neither decides whether a specific tool call with specific arguments should execute, which is the gap an action-level policy layer fills.
How do I secure code execution in NeMo Agent Toolkit?
Since 1.9.0 code execution runs against a remote sandbox server, with Piston as the documented option. Run that server in an isolated network segment with no route to production data, do not leave it on a plain HTTP endpoint reachable from other workloads, cap timeouts and output size, and require approval before model-written code touches anything outside the sandbox.
Can NeMo Agent Toolkit agents access other users' data?
The toolkit isolates per-user workflow state by the resolved user ID, and since 1.9.0 the model can no longer pick the user ID for memory tools. Because identity resolution trusts the incoming credential, a request with a forged token can still claim another identity unless a proxy validates tokens first. Retrievers and database tools need their own scoping.
Does AgentShield work with NeMo Agent Toolkit?
Yes. AgentShield sits on the action path between the workflow and the tools it calls, so it applies the same permissions, approval gates, injection screening and audit whether the workflow is built on the toolkit directly or on LangChain, LlamaIndex or CrewAI underneath it. It does not replace the toolkit or its profiler.
More use cases