Latitude
Open-source agent engineering platform for AI observability, issue detection, and evaluation - described as Sentry for AI agents and LLMs
4K+
GitHub Stars
100%
Trace Search
MIT
License
Overview
Latitude is an open-source agent engineering platform that functions as "Sentry, but for AI agents and LLMs." It helps developers understand what will break next in their AI applications and fix issues before users notice. The platform provides issue-centric observability where failed traces are automatically grouped into tracked issues with status, size, and trend analysis. Latitude features human-aligned evaluations built automatically from team judgments, agent-native traces that capture multi-turn sessions and tool calls, and semantic search across 100% of traces without sampling.
The Verdict
Who Should Use Latitude?
Best For
- Teams building production AI agents
- Developers needing LLM error tracking
- Organizations wanting self-hosted observability
- Claude Code and Vercel AI SDK users
- Teams with human evaluation workflows
Not Ideal For
- Simple single-prompt applications
- Teams without production LLM workloads
- Those needing only prompt versioning (try PromptLayer)
- Pure evaluation focus (try Promptfoo)
What's Great
- True open-source with MIT license
- Issue-centric approach groups failures automatically
- 100% trace searchability without sampling
- Human-aligned evals track drift from human judgment
- Native Claude Code integration
- Provider-agnostic (OpenAI, Anthropic, Bedrock, etc.)
- Self-hosting option available
Watch Out For
- Relatively new platform (founded 2024)
- V2 still in development (not stable release)
- Smaller community than established alternatives
- Limited third-party integrations vs. LangSmith
- Self-hosting requires infrastructure management
Pricing
Free
$0
Start free with core observability
Pro
Usage-based
Scale as your agents grow
Self-Hosted
Free (OSS)
Full control, your infrastructure
Enterprise
Custom
Commercial license, dedicated support
View all features & details
Core Features
- Issue discovery and clustering
- Agent-native trace visualization
- Multi-turn session tracking
- Tool call monitoring
- Semantic search across all traces
- Human-aligned evaluation generation
- Evaluation alignment tracking (MCC, coverage)
- OTel ingest support
Integrations
- OpenAI
- Anthropic / Claude
- AWS Bedrock
- Vercel AI SDK
- LangChain
- Claude Code (dedicated package)
- Any OTLP-compatible backend
Languages & SDKs
- TypeScript / JavaScript (npm)
- Python
- Go
- OpenTelemetry passthrough
Deployment Options
- Latitude Cloud (managed)
- Self-hosted (Docker)
- LGPL-3.0 open source
- Commercial license available
How It Compares
| Feature | Latitude | LangSmith | Promptfoo | Helicone |
|---|---|---|---|---|
| Open Source | Yes (MIT) | No | Yes | Partial |
| Issue Clustering | Automatic | Manual | No | No |
| Human-Aligned Evals | Yes | Basic | Yes | No |
| Semantic Search | 100% traces | Sampled | No | Limited |
| Self-Hosting | Full OSS | Enterprise only | Yes | Limited |
| Claude Code Integration | Native package | Via LangChain | Manual | Manual |
| Pricing Start | Free | Free tier | Free | Free tier |
| Best For | Agent debugging | LangChain teams | Eval testing | Cost tracking |
Resources
Roadmap Highlights
- OTel ingest, traces, sessions - Done
- Issue discovery + clustering - Done
- Issue-to-eval generation - Done
- Semantic search - Done
- Potential issues tracking - Upcoming
- Stable v2 release - Upcoming
User Reviews
Loading reviews...