SGLang iconSGLang

Open-source LLM, vision and diffusion serving engine from LMSYS, built for agentic workloads, large-scale serving and RL rollouts

38.7k+ GitHub Stars
v0.5.23 Latest (Nov 2, 2026)
Apache-2.0 License

Overview

SGLang is an open-source serving framework for large language models, vision-language models and, through SGLang Diffusion, image and video models. Its README describes it as optimized for agentic workloads, large-scale serving and reinforcement-learning rollouts, and it is hosted by LMSYS, a non-profit research organization. Its RadixAttention prefix cache makes it strong when many requests share long prompts, which is common in agents and multi-turn chat. It supports tensor and expert parallelism across hosts, speculative decoding (with SpecForge to train draft models), and SGLang Omni for speech. Hardware coverage is wide: NVIDIA H100, B200 and GB300, AMD MI300 to MI355, Google TPU, Intel, Apple Silicon and Huawei Ascend. RL frameworks such as verl, slime and AReaL use it for rollouts, and it plugs into llm-d, Ray Serve and NVIDIA Dynamo. In January 2026 the core team spun out RadixArk, which announced a $100M seed led by Accel in May 2026 and has started selling hosted services while keeping the engine open source. Its closest rival is vLLM; see also TensorRT-LLM.

The Verdict

Who Should Use SGLang?

Best For

  • Agent and chat workloads with heavy shared prefixes
  • High-concurrency serving where tail latency matters
  • RL teams generating rollouts at scale

Not Ideal For

  • Running a model on a laptop (use a local runtime)
  • Teams that want a managed API with no ops work

What's Great

  • Much lower P99 time-to-first-token than vLLM at high concurrency in one user's test
  • Prefix caching shines when many requests share context
  • Broad hardware support, from NVIDIA and AMD to TPU and Ascend

Watch Out For

  • Raw throughput slightly below vLLM in one production test
  • vLLM tends to get day-one support for new models first
  • Self-hosting means you run GPUs, scaling and upgrades yourself

Pricing

View all features & details

Key Features

  • RadixAttention prefix caching
  • Tensor and expert parallelism across hosts
  • Speculative decoding with SpecForge draft models
  • SGLang Diffusion for image and video generation
  • SGLang Omni for text-to-speech and speech recognition
  • OpenAI-compatible server
  • Integrations with llm-d, Ray Serve and NVIDIA Dynamo

Project and Company

  • Hosted by LMSYS (non-profit)
  • RadixArk spin-out reported January 2026 at about a $400M valuation
  • $100M seed led by Accel, with Spark, NVentures and AMD (May 2026)

How It Compares

Feature SGLang vLLM TensorRT-LLM
License Apache-2.0 Apache-2.0 Apache-2.0
Strength Shared-prefix, high-concurrency latency Ecosystem and day-one model support Peak NVIDIA performance
Hardware NVIDIA, AMD, TPU, Intel, Ascend, Apple NVIDIA, AMD, Intel, CPU, plugins NVIDIA only
Company behind it RadixArk Inferact NVIDIA

Sources: GitHub · sglang.io · TechCrunch (Jan 2026) · SiliconANGLE (May 2026)

Is this your tool? Add a badge