Datadog Agent Observability (formerly LLM Observability) iconDatadog Agent Observability (formerly LLM Observability)

Datadog's AI agent and LLM monitoring with tracing, evals, experiments and datasets, linked to your full Datadog stack

40K Free LLM Spans/Month
$160/mo Pro Plan From
Free Tool and Retrieval Spans

Overview

Datadog's product for monitoring LLM apps and AI agents, now called Agent Observability (previously LLM Observability), traces every step of an agent run: LLM calls, tool calls, retrieval and workflow steps, with latency, token usage, cost and errors. It adds built-in and custom evaluations, human review and annotation, versioned datasets made from production traces, and experiments that compare prompts, models and configurations side by side.

The main reason to choose it is integration: agent traces connect to the rest of Datadog, so you can follow a slow answer from the user's browser session through your backend services and into the model call, in the same tool your SRE team already uses. Datadog moved to a simpler pricing model with a free tier and per-LLM-span billing, where tool, workflow and retrieval spans are free.

The Verdict

Who Should Use Datadog Agent Observability?

Best For

  • Companies already standardized on Datadog
  • Correlating agent behavior with backend and frontend performance
  • Platform and SRE teams owning AI in production
  • Enterprises that want one vendor for all monitoring

Not Ideal For

  • Startups not using Datadog for anything else
  • Teams that need self-hosting or open source
  • Very high LLM call volumes on a tight budget

What's Great

  • End-to-end correlation with APM, logs and real user monitoring
  • Free tier and billing only on LLM spans
  • Evaluations, datasets and experiments in one place
  • Supports OpenAI, Anthropic, Gemini, Bedrock and major frameworks
  • Enterprise-grade security and compliance

Watch Out For

  • Datadog bills add up across products
  • Less specialized eval workflows than eval-first tools
  • No self-hosting
  • Retention beyond 15 days is a paid add-on

Pricing

View all features & details

Key Features

  • Full-stack agent tracing
  • Latency, token, cost and error monitoring
  • Built-in and custom evaluations
  • Datasets from production traces
  • Experiments comparing prompts and models
  • Human review and annotation

Integrations

  • OpenAI, Anthropic, Gemini, Vertex AI, Bedrock
  • LangChain, CrewAI, Pydantic AI, Strands Agents
  • Python, Node.js and Java SDKs
  • OpenTelemetry and HTTP API

How It Compares

Feature Datadog Langfuse Braintrust Arize AX
Full-stack APM correlation Yes No No No
Free tier 40K LLM spans 50K units 1 GB data 25K spans
Self-hosting No Yes Enterprise Enterprise
Eval depth Good Good Best Good

Is this your tool? Add a badge