Label Studio
Open-source data labeling and AI evaluation platform by HumanSignal: vision, text, audio, time series and LLM evals, self-hosted or cloud
Overview
Label Studio's line is "Label any data. Evaluate any AI.": it is "the open source platform for data labeling, AI evaluation, and human-in-the-loop workflows." It is built by HumanSignal (formerly Heartex) and licensed Apache-2.0, so you can self-host the Community Edition for free or pay for HumanSignal's cloud and enterprise plans. One configurable interface covers computer vision, documents and NLP (including chatbots), audio and speech, time series and multimodal data. For LLM work it adds evaluation with benchmarks, rubrics, side-by-side comparison, RLHF preference collection, RAG relevance and agent trace review, and the paid tiers add Prompts for comparing model output against ground truth. It plugs into ML pipelines through an API, a Python SDK, webhooks and model connections for AI-assisted labeling. Latest stable release: 1.23.2, September 29, 2026.
The Verdict
Who Should Use Label Studio?
Best For
- Teams that want free, self-hosted labeling with no lock-in
- Projects mixing many data types in one tool
- LLM teams collecting human evaluations and preference data
Not Ideal For
- Teams that need review queues, consensus, roles and SSO without paying for Enterprise
- Buyers who want a fully managed labeling workforce by default
- Non-technical users who cannot set up and configure templates
What's Great
- Open source (Apache-2.0) with a free, self-hostable Community Edition
- Very flexible: configurable templates and tags for almost any data type
- SDK, API and webhooks fit into existing ML pipelines
- LLM and agent evaluation built in
Watch Out For
- Review workflows, role management, SSO and quality review are Enterprise features
- Learning curve and setup effort; heavier on resources at scale
- Too few G2 reviews to judge satisfaction
Pricing
View all features & details
Data Types
- Computer vision
- Documents and NLP, including chatbot conversations
- Audio and speech
- Time series
- Multimodal and LLM or agent outputs
AI Evaluation
- Benchmarks and rubrics
- Side-by-side model comparison
- RLHF preference collection
- RAG relevance and agent trace review
- Prompts (Starter Cloud and Enterprise)
Real-World Usage
Named Users
- Logos on labelstud.io: Cloudflare, NVIDIA, Meta, IBM, Intel, SRI, SPI Global
Project Activity
- Large GitHub following (see badge), about 3.8k forks
- 1.22.0 (January 2026), 1.23.0 (March 2026), 1.23.2 (September 2026), plus nightly builds
How It Compares
| Feature | Label Studio | Labelbox | Argilla | Encord |
|---|---|---|---|---|
| License | Apache-2.0 | Commercial | Apache-2.0 | Commercial |
| Entry price | Free self-hosted; $99/mo cloud | Free 500 LBU a month | Free | Contact sales |
| Data types | Vision, text, audio, time series, LLM | Multimodal, LLM, agents | Text and LLM data | Vision, video, DICOM, LiDAR, audio, text |
| 2026 status | Active | Active | Maintenance only | Active |
Sources: labelstud.io · Labelbox limits · Argilla GitHub · Encord supported data
Is this your tool? Add a badge