How to Monitor AI Agents in Production

To monitor AI agents, instrument every action: each tool or MCP call, its arguments, the target resource, the human identity behind it, and the policy verdict. Record the full action chain per session in an append-only log. Baseline normal behavior in observe-only mode first, then alert on deviations like new tools, denied-action spikes, or unusual data egress.

Most teams monitor agents like a web service: traces, token counts, p95 latency, prompt and completion logged. Then an incident happens, someone asks "what did the agent touch in the customer database at 2am, and who authorized it," and the dashboards have nothing to say. For the product view, see AI agent monitoring. This piece is the plumbing.

What should you monitor in an AI agent?

Monitor actions, not tokens. For every tool or MCP call, capture the tool name, the full arguments, the target resource (which table, which repo, which mailbox), the identity of the human or service the agent acts for, the policy decision (allow, block, or hold), approval events, data classifications on egress, and the ordered action chain for the session.

The key insight is the unit of monitoring. For a chatbot it's the request. For an LLM feature it's the token. For an agent, the unit is the action: tool call plus resource plus verdict. An agent that burns 40,000 tokens reasoning about a refund policy and does nothing is boring. An agent that spends 300 tokens and issues a refund call against production billing is the entire security story. Token counts cannot tell those apart. Action logs can.

Two fields carry most of the weight. Who: the human principal, not just "agent-7". No attributable person, no accountability chain. Where: the specific resource. "Called the database tool" is useless. "Ran a SELECT against customers.pii_view, returned 12,400 rows" is an incident report.

How is AI agent monitoring different from LLM observability?

LLM observability answers "is it working and what did it cost." It tracks latency, tokens, spend, traces, and prompt/response pairs. AI agent security monitoring answers "what did it do and was it allowed." It tracks which tool ran on which resource under whose permission, the policy verdict, approval status, data egress, and immutable attribution. You need both. They aren't substitutes.

Your observability stack is good at its job. It just wasn't built to answer questions from a lawyer or an incident commander.

Step 1: Inventory the agent's action surface

List every tool the agent can reach: every MCP server, every function in the tool schema, every credential it holds. Teams are consistently surprised. An agent that "just reads Jira" turns out to hold a token with write scope, or reaches a shared MCP gateway fronting eleven other systems. You cannot monitor a surface you haven't mapped, and mapping it usually shortens it. Our writeup on least-privilege tool permissions covers trimming that list.

Step 2: Instrument at the action boundary, not inside the agent

Instrument inside the framework (LangChain callbacks, CrewAI hooks, custom decorators) and you get logs covering only the paths you remembered to decorate, which break whenever someone swaps frameworks or an agent spawns a subagent. Intercept at the boundary instead: a proxy or gateway in front of tool and MCP traffic. Every call passes one chokepoint, gets recorded, gets a verdict. That's what makes the record complete rather than best-effort, and why stack-neutral placement (OpenAI, Anthropic, LangChain, CrewAI, MCP) beats framework-native hooks.

Attach identity here too: the session carries the principal it acts for, and that principal lands in every record. Retrofitting attribution later is usually impossible.

Step 3: Run observe-only and build a baseline

Don't enable enforcement first. Run observe-only for a full business cycle (month-end close, a release week, whatever your seasonality is) and learn normal: which tools each agent actually uses versus which it can use, typical volume per action type, time-of-day distribution, egress destinations, and the usual shape of action chains.

Same discipline teams already apply when they monitor their data pipelines for freshness, volume, and schema anomalies: learn the shape of normal before flagging abnormal. Skip the baseline and your first week of enforcement blocks legitimate work, your team loses trust, and someone turns it off.

Step 4: Decide what alerts and what only gets recorded

Record everything. Alert on almost nothing. This decision determines whether your monitoring survives an on-call rotation.

Alert-worthy:

• An agent calls a tool it has never called before.
• A spike in denied actions from one agent or session. Denials are cheap individually, diagnostic in aggregate.
• Data egress to a destination not in the baseline.
• Read volume far outside the normal envelope (the 12,000-row SELECT from a service that normally reads 20).
• Repeated injection detections tracing back to one source document or feed.
• Out-of-hours activity for agents with a predictable schedule.
• An action chain diverging from the task given: asked to summarize a ticket, now calling the email tool.
• An approval granted by an unexpected principal.

Record-only, never paged: routine allowed actions, normal read volumes, individual denials within baseline rates, latency and token data, in-pattern approvals. The temptation is to page on every block because a block feels like an attack. It usually isn't. It's a misconfigured scope. Let the aggregate tell you.

How do you detect a compromised AI agent?

Look for behavioral divergence, not payload signatures. A compromised agent reveals itself through a tool it has never called, an action chain that no longer matches its assigned task, a jump in denied actions, unusual read volume, egress to an unfamiliar destination, or injection detections clustering on one document. Individually these are noise. Correlated on one session, they're an incident.

The tell that matters most is task divergence. Legitimate agents do boring, repetitive things. An agent given "triage this support ticket" that suddenly reads the credentials table and calls an outbound HTTP tool got redirected, usually by injected content in the ticket itself. Detection is only step one: our guide on containing a compromised AI agent covers the response side.

Step 5: Make the log immutable and keep it long enough

Mutable logs aren't evidence. If the system that writes the record can silently rewrite it, the record proves nothing to an incident reviewer and nothing to an auditor. Append-only, tamper-evident storage is the baseline, which is why our immutable audit trail exists.

Retention is a real decision, not a default. Breaches get discovered well after they start, so a 7-day window means reconstructing the important part from nothing. The EU AI Act's obligations for high-risk AI systems apply from August 2, 2026, including automatic logging of events over the system's lifetime and record-keeping that supports traceability. If your agents touch anything in scope, your monitoring is the evidence layer. Specifics in AI agent audit requirements; the program view sits under AI governance.

How do you monitor AI agents in production?

Route all tool and MCP traffic through a policy checkpoint that records every action, attaches the human principal, and returns a verdict. Baseline in observe-only for one full business cycle, then enable enforcement on the highest-risk tools first. Alert on behavioral divergence, record everything else immutably, and review denied actions weekly.

Order matters. "Highest-risk" means anything that writes to production, moves money, sends external email, or reads regulated data. Gate those with approvals rather than hard blocks so humans stay in the loop while you tune, then expand outward as false positives drop. Every denial you review is either a scope that's too tight or an agent doing something it shouldn't.

The real-time agent monitoring view and the policy engine should be one system, because a verdict you can see but not enforce is just a nicer dashboard.

What are the best AI agent monitoring tools?

Two categories, and you likely need one from each. LLM observability tools (LangSmith, Langfuse, Datadog LLM Observability, Arize, plus tracing built into most frameworks) handle cost, latency, and quality. Agent security monitoring and control planes, including Agentshield, handle action-level records, identity attribution, policy verdicts, approval gates, and immutable audit. Judge them on different criteria:

• Completeness. Every action, or a sample? Sampling is fine for performance data and disqualifying for security data.
• Attribution. Can it name a human principal, or does it stop at the service account?
• Stack neutrality. Will it survive the move from LangChain to raw MCP? Framework-coupled monitoring rots.
• Enforcement. If it detects something, can it stop it, or only file a report?

Common mistakes

• Logging only prompts and responses. The most common error by far. You get a transcript of what the agent said and no record of what it did. Those diverge constantly, and the second is what matters in an incident.
• Mutable logs. A log your incident could have edited is not a log of your incident.
• No human attribution. "Agent 7 deleted the records" is not an answer. "Agent 7, acting for jsmith, deleted the records under policy exception 14" is.
• Sampling. Your APM samples because full capture is expensive and the value is statistical. Security value isn't statistical.
• Monitoring without enforcement. Which brings us to the honest part.

Monitoring is detection, not prevention

Being direct, because the industry mostly isn't: monitoring tells you what happened. It doesn't stop anything. A perfect audit trail of an agent exfiltrating your customer list is a very well-documented breach. It still earns its keep three ways: the baseline that makes enforcement possible, the evidence trail after the fact, and the slow-burn problems no policy would have blocked because each action looked legitimate alone. Those are real. They're also not prevention.

Pair it with enforcement at the same chokepoint: least-privilege tool permissions, approval gates on dangerous actions, data-loss checks on egress, injection detection on untrusted input. One instrumentation point, one side watching, the other deciding. The sequence is in how to secure AI agents.

The takeaway

Monitor actions, not tokens. Instrument at the tool and MCP boundary so capture is complete rather than best-effort, attach a human principal to every action, record the full chain immutably, baseline in observe-only before enforcing anything, and alert on behavioral divergence rather than raw volume. Then add enforcement, because a record of the breach is not a defense against it.

Agentshield does both at one chokepoint: action-level records with immutable attribution, plus the policy engine, permissions, and approval gates that act on them. If you only need cost and latency, an observability tool is cheaper and better at that job. If you need to answer "what did the agent do, on whose authority, and was it allowed," that's what we built. Start with AI agent monitoring.