AgentOps iconAgentOps

Observability for AI agents with session replays, cost tracking and failure analysis, via an MIT-licensed SDK for CrewAI and more

5.8k+ GitHub Stars
MIT SDK License
$40/mo Pro Plan

Overview

AgentOps is an observability platform built around AI agent sessions. After adding its SDK (two lines of Python), every agent run is recorded as a session you can replay step by step: the LLM calls, tool calls, errors and handoffs between agents, along with token usage and cost. That makes it easier to see where a multi-step agent went wrong, how much a run cost, and whether a change made things better or worse.

The SDK is open source under the MIT license and integrates natively with agent frameworks such as CrewAI, AutoGen/AG2, the OpenAI Agents SDK, LangChain and LlamaIndex, plus most major model providers. The hosted dashboard has a free tier, a Pro plan and an enterprise plan with self-hosting and compliance options.

The Verdict

Who Should Use AgentOps?

Best For

  • Developers building multi-agent systems with CrewAI or AutoGen
  • Replaying agent sessions to debug failures
  • Tracking cost per agent run
  • Small teams wanting quick setup and a low price

Not Ideal For

  • Teams needing deep evaluation and dataset workflows (see Braintrust)
  • Fully self-hosted open-source deployments on a budget
  • Non-agent LLM apps

What's Great

  • Two-line setup with native agent framework support
  • Session replay shows every step an agent took
  • Cost and token tracking per session
  • Affordable Pro tier with no event cap
  • Open-source SDK

Watch Out For

  • Free tier capped at 5,000 events per month
  • Self-hosting only on enterprise plans
  • Evaluation features are lighter than eval-first platforms
  • No commits to the SDK repo since June 2026

Pricing

Prices as reported by third-party reviews in 2026; confirm on the official site.

View all features & details

Key Features

  • Session replays with step-by-step timelines
  • LLM cost and token tracking
  • Error and failure analysis
  • Multi-agent interaction views
  • Session export

Integrations

  • CrewAI, AutoGen / AG2
  • OpenAI Agents SDK
  • LangChain, LlamaIndex
  • OpenAI, Anthropic, Gemini, Mistral and more

How It Compares

Feature AgentOps Laminar Langfuse LangSmith
Agent session replay Yes Yes Traces Traces
Open source SDK (MIT) Full platform Full platform No
Self-hosting Enterprise Yes Yes Enterprise
Paid entry price $40/mo Usage-based $29/mo $39/seat

Is this your tool? Add a badge