Label Studio iconLabel Studio

Open-source data labeling and AI evaluation platform by HumanSignal: vision, text, audio, time series and LLM evals, self-hosted or cloud

28.6k+ GitHub Stars
v1.23.2 Latest Release (Sep 2026)
$99/mo Starter Cloud

Overview

Label Studio's line is "Label any data. Evaluate any AI.": it is "the open source platform for data labeling, AI evaluation, and human-in-the-loop workflows." It is built by HumanSignal (formerly Heartex) and licensed Apache-2.0, so you can self-host the Community Edition for free or pay for HumanSignal's cloud and enterprise plans. One configurable interface covers computer vision, documents and NLP (including chatbots), audio and speech, time series and multimodal data. For LLM work it adds evaluation with benchmarks, rubrics, side-by-side comparison, RLHF preference collection, RAG relevance and agent trace review, and the paid tiers add Prompts for comparing model output against ground truth. It plugs into ML pipelines through an API, a Python SDK, webhooks and model connections for AI-assisted labeling. Latest stable release: 1.23.2, September 29, 2026.

The Verdict

Who Should Use Label Studio?

Best For

  • Teams that want free, self-hosted labeling with no lock-in
  • Projects mixing many data types in one tool
  • LLM teams collecting human evaluations and preference data

Not Ideal For

  • Teams that need review queues, consensus, roles and SSO without paying for Enterprise
  • Buyers who want a fully managed labeling workforce by default
  • Non-technical users who cannot set up and configure templates

What's Great

  • Open source (Apache-2.0) with a free, self-hostable Community Edition
  • Very flexible: configurable templates and tags for almost any data type
  • SDK, API and webhooks fit into existing ML pipelines
  • LLM and agent evaluation built in

Watch Out For

  • Review workflows, role management, SSO and quality review are Enterprise features
  • Learning curve and setup effort; heavier on resources at scale
  • Too few G2 reviews to judge satisfaction

Pricing

View all features & details

Data Types

  • Computer vision
  • Documents and NLP, including chatbot conversations
  • Audio and speech
  • Time series
  • Multimodal and LLM or agent outputs

AI Evaluation

  • Benchmarks and rubrics
  • Side-by-side model comparison
  • RLHF preference collection
  • RAG relevance and agent trace review
  • Prompts (Starter Cloud and Enterprise)

Real-World Usage

Named Users

  • Logos on labelstud.io: Cloudflare, NVIDIA, Meta, IBM, Intel, SRI, SPI Global

Project Activity

  • Large GitHub following (see badge), about 3.8k forks
  • 1.22.0 (January 2026), 1.23.0 (March 2026), 1.23.2 (September 2026), plus nightly builds

How It Compares

Feature Label Studio Labelbox Argilla Encord
License Apache-2.0 Commercial Apache-2.0 Commercial
Entry price Free self-hosted; $99/mo cloud Free 500 LBU a month Free Contact sales
Data types Vision, text, audio, time series, LLM Multimodal, LLM, agents Text and LLM data Vision, video, DICOM, LiDAR, audio, text
2026 status Active Active Maintenance only Active

Sources: labelstud.io · Labelbox limits · Argilla GitHub · Encord supported data

Is this your tool? Add a badge