AI Agent Observability: How to Debug Agents in Production

TL;DR

• A tool can return HTTP 200 and still fail in the response body. Only the trace shows whether the model saw that failure.
• Record enough per step to locate the break: prompt and app versions, tool arguments and results, retrieved document IDs, context passed between agents.
• Debug by finding where the agent's answer diverges from the recorded results. Then keep the case as a regression test — failure, success, and unknown-outcome timeout.
• Traces lie by omission: head-based sampling drops runs before the error appears, and no trace proves an external operation actually happened.

Debugging an AI agent starts with tracing how it carried out the task. A refund tool can return HTTP 200 and still report a failure in the response body. If the agent overlooks that message and tells the customer the refund went through, monitoring status codes alone won’t flag the mistake.

AI agent observability helps you investigate these failures. A trace records the run, letting you compare the agent’s response with the results of its tool calls. In this example, it helps establish whether the model received the failure message. That determines where to look next: the code passing the tool’s result to the model, or how the model handled that result.

What is AI agent observability in production?

AI agent observability is the practice of collecting and analyzing data about how an agent performs tasks. It connects outcomes to the agent’s actions so teams can investigate what went wrong.

Four related processes help teams monitor agents and investigate failures:

• Monitoring tracks changes in performance, such as rising task failure rates or longer execution times.
• Tracing lets you inspect an individual run to see what data the model received and what the tools returned.
• Evaluation checks the outcome against defined criteria, including whether the agent completed the user’s request.
• Debugging uses these records and evaluation results to find the cause of a failure and test a fix.

‍

Agent observability is one area of AI observability. LLM observability focuses on calls to the model and its responses. Agent workflows also require visibility into tool execution and the context passed between steps. Errors in these transitions may not be apparent from the final response. How thoroughly you can investigate them depends on which steps and data the system records.

What should you record in an AI agent trace?

A trace links the recorded steps of a task. Each span describes an operation, including its start and end times and relevant metadata. AI agent tracing uses distributed tracing to keep these records connected, including across service boundaries.

What you record depends on the operation. The table below shows which details help you investigate failures at each step.

Tool arguments and context can contain sensitive information, so define what can be recorded. OpenTelemetry recommends opt-in capture for full prompts and responses. Large or sensitive content can be stored separately, with references added to the spans.

How do you instrument an AI agent for production tracing?

Your tracing setup should let you follow a request from the agent invocation through to the results of its tool calls. Work through these steps:

1. Set up the tracing SDK and configure data export. Start with the official OpenTelemetry configuration example for Python, which sends spans to an OTLP endpoint. Point it to your telemetry service and create a span around the agent invocation to connect its nested operations.
2. Enable LLM tracing through an integration supported by your model client or framework. Most LLM observability tools capture common model calls automatically, but custom tools and retrieval operations may still need manual instrumentation.
3. Propagate trace context between services. It carries the identifiers needed to connect downstream operations to the originating trace. HTTP requests typically use the traceparent header, as described in the OpenTelemetry documentation. Check context propagation through queues and background tasks separately.
4. Define what to record. Add the deployment environment and application version so you can distinguish test runs from production traffic. Specify which fields can be stored and configure redaction of sensitive values before export.
5. Validate the setup with test data. Run one successful request and another in which a tool fails. Open both traces and check the nesting of operations and the recorded results. The failed run should show what the tool reported, even if the top-level HTTP request returned a 200 status code.

Teams that are still designing an agent can work with Tensorway on AI agent development

How do you debug an AI agent that returns the wrong result?

Start with the trace of the failed run. Find it using the request or run ID, then compare the agent’s response with the results of the operations it relied on. Look for where the response diverges from the recorded results.

How do you find the failing step?

In the refund example, the agent told the customer the refund went through, but the tool returned  “success: false”. Open the tool span and inspect its arguments and result. Then check the input to the next model call, if it was captured. Does it include the failure message?

If the message was dropped or replaced with an empty response, inspect the code that passes tool results to the model. If the failure appears in the model’s input, examine the instructions and how the agent handles failed operations. A tool call finishing does not, by itself, mean the refund succeeded.

For workflows involving several agents, also check the context passed between them. In the deal-sourcing system built by Tensorway, separate agents handle research, evaluation, and report preparation. When investigating a failure in this type of architecture, check which findings from earlier steps reached the agent that produced the final response.

How do you check that the fix works?

Reproduce the failure in a test environment using a recorded or mocked tool response. Make sure the test does not issue a real refund. Record the model, prompt, and application versions, along with the available initial context.

The fix depends on what caused the error. You may need to restore the missing error message, change how tool results are handled, or revise the instructions. A refund confirmation must be backed by a successful result from the payment system. If the status is unknown, the agent should say so.

Test a tool failure, a successful refund, and a timeout where the outcome is unknown. Define the expected behavior for each case. For example, after an explicit failure, the agent must not tell the customer their money has been refunded.

Add these cases to your agent evaluation dataset. Langfuse lets you create test cases from production traces. Run them after changes to the prompt, model, or code to check whether the error has returned.

Which metrics reveal AI agent failures in production?

A single metric rarely explains why an agent has started performing worse. AI agent monitoring needs metrics that connect the final outcome to the individual steps in the run.

At the model level, LLM monitoring helps track token usage, latency, model errors, and changes in response quality. Agent-level metrics add the context needed to understand how these model calls affect the entire workflow.

At a minimum, track these groups of metrics:

• Task outcome. The share of successfully completed requests, refusals, and responses that require human intervention. Define quality criteria for the process itself, such as whether the amount is correct, the answer is complete, or the agent followed the relevant business rules.
• Tool performance. The number of errors, timeouts, and retries for each tool. Track cases where the HTTP request returned a 200 status code but the response body contained an error.
• Agent loop. The number of steps, total run time, and share of runs that stop after reaching a limit. A sudden increase in iterations may point to poor planning, repeated tool calls, or a missing stopping condition.
• Resource use. Input and output tokens, model-call costs, and response time. Count tokens at the model-span level so the same usage is not added again from a parent trace.

Break these metrics down by model, prompt, application version, request type, and environment. This helps you distinguish a general decline from a problem affecting one particular scenario.

How do you choose an AI agent observability tool?

The right agent observability tools should help you connect technical telemetry with the decisions you need to make about the agent’s performance. If your team only needs to see errors and latency, an existing APM system with custom spans may be enough. More complex agent workflows need a clearer view of model calls, tools, retrieval, and context passed between agents.

Before choosing a platform, check five things:

1. Coverage for your stack. Does it provide integrations for your model provider, framework, queues, and custom tools? Automatic instrumentation does not guarantee that every operation you need will be captured.
2. Data format and interoperability. OpenTelemetry makes it easier to send spans between systems, but using the same transport does not guarantee that platforms will display attributes in the same way. LangSmith supports OpenTelemetry tracing for LangChain and custom applications, while Phoenix uses OpenInference and can convert other conventions through span processors. LangSmith documents automatic and manual instrumentation, while Phoenix explains how it translates different formats into OpenInference.
3. Search and filtering. You will need filters for application version, environment, customer, error type, and run ID.
4. Outcome evaluation. Check whether you can create datasets from production traces, run regression tests, and store evaluation results alongside the trace.
5. Data handling and cost. Find out where prompts, responses, and tool arguments are stored, how sensitive data is redacted, and how much it costs to retain a growing volume of telemetry.

For a small team using one framework, a native integration may be the simplest place to start. If the system spans several services or programming languages, build around a shared OpenTelemetry layer and verify how the chosen backend handles GenAI attributes.

What can make a production trace incomplete?

A trace contains only the operations the system actually recorded. Data can be missing because an integration was disabled, context propagation failed, or sampling removed the trace. With head-based sampling, the decision to keep a trace is made at the start of the run, so the system cannot know whether an error will appear later. OpenTelemetry recommends tail sampling when you need to retain traces with errors or high latency.

Problems can also appear after an SDK or framework upgrade. Automatic instrumentation may stop covering some API calls after a version change. In a public Langfuse discussion, a developer described cases where some API calls did not appear in the traces. The report does not establish one universal cause, but it shows why teams should test instrumentation coverage after configuration changes.

Even a complete trace cannot prove that an external operation actually took place. Data from a payment provider, CRM, or another service still needs to be checked against the resulting business state. Prompt and model-output capture may also be limited by privacy requirements. In that case, store only approved fields or references to a protected store, and make it clear which data is absent from the trace.

Conclusion

AI agent observability replaces guesswork with a way to inspect a specific run. A trace shows what the agent received, which tools it called, and where the result began to diverge from the expected outcome. Turning that information into a reliable production process requires careful instrumentation, outcome checks, and regression tests.

If you need help designing or improving an agent workflow, Tensorway develops AI agents for production use cases.

‍