SGLang
Open-source LLM, vision and diffusion serving engine from LMSYS, built for agentic workloads, large-scale serving and RL rollouts
Overview
SGLang is an open-source serving framework for large language models, vision-language models and, through SGLang Diffusion, image and video models. Its README describes it as optimized for agentic workloads, large-scale serving and reinforcement-learning rollouts, and it is hosted by LMSYS, a non-profit research organization. Its RadixAttention prefix cache makes it strong when many requests share long prompts, which is common in agents and multi-turn chat. It supports tensor and expert parallelism across hosts, speculative decoding (with SpecForge to train draft models), and SGLang Omni for speech. Hardware coverage is wide: NVIDIA H100, B200 and GB300, AMD MI300 to MI355, Google TPU, Intel, Apple Silicon and Huawei Ascend. RL frameworks such as verl, slime and AReaL use it for rollouts, and it plugs into llm-d, Ray Serve and NVIDIA Dynamo. In January 2026 the core team spun out RadixArk, which announced a $100M seed led by Accel in May 2026 and has started selling hosted services while keeping the engine open source. Its closest rival is vLLM; see also TensorRT-LLM.
The Verdict
Who Should Use SGLang?
Best For
- Agent and chat workloads with heavy shared prefixes
- High-concurrency serving where tail latency matters
- RL teams generating rollouts at scale
Not Ideal For
- Running a model on a laptop (use a local runtime)
- Teams that want a managed API with no ops work
What's Great
- Much lower P99 time-to-first-token than vLLM at high concurrency in one user's test
- Prefix caching shines when many requests share context
- Broad hardware support, from NVIDIA and AMD to TPU and Ascend
Watch Out For
- Raw throughput slightly below vLLM in one production test
- vLLM tends to get day-one support for new models first
- Self-hosting means you run GPUs, scaling and upgrades yourself
Pricing
View all features & details
Key Features
- RadixAttention prefix caching
- Tensor and expert parallelism across hosts
- Speculative decoding with SpecForge draft models
- SGLang Diffusion for image and video generation
- SGLang Omni for text-to-speech and speech recognition
- OpenAI-compatible server
- Integrations with llm-d, Ray Serve and NVIDIA Dynamo
Project and Company
- Hosted by LMSYS (non-profit)
- RadixArk spin-out reported January 2026 at about a $400M valuation
- $100M seed led by Accel, with Spark, NVentures and AMD (May 2026)
How It Compares
| Feature | SGLang | vLLM | TensorRT-LLM |
|---|---|---|---|
| License | Apache-2.0 | Apache-2.0 | Apache-2.0 |
| Strength | Shared-prefix, high-concurrency latency | Ecosystem and day-one model support | Peak NVIDIA performance |
| Hardware | NVIDIA, AMD, TPU, Intel, Ascend, Apple | NVIDIA, AMD, Intel, CPU, plugins | NVIDIA only |
| Company behind it | RadixArk | Inferact | NVIDIA |
Sources: GitHub · sglang.io · TechCrunch (Jan 2026) · SiliconANGLE (May 2026)
Is this your tool? Add a badge