Guidance
MIT library for constrained LLM generation (regex, grammars, JSON schema), powered by llguidance, the engine behind structured outputs in llama.cpp and vLLM
Overview
Guidance is "a guidance language for controlling large language models", first released by Microsoft and now maintained in the guidance-ai GitHub organization. Instead of prompting and hoping, you write Python programs that mix free generation with hard constraints: select from options, match a regex, follow a context-free grammar or a JSON schema. Where the backend exposes the model's tokens (local models such as Transformers and llama.cpp), the constraints are enforced token by token, so the output fits the format. Its engine, llguidance, is a separate Rust library that reached 1.0 in June 2025; the project lists OpenAI's JSON-schema structured outputs, llama.cpp, vLLM, SGLang, Chromium, mistral.rs and onnxruntime-genai as users, and claims about 1.5 ms of mask computation per token for a typical JSON schema. Recent releases (0.3.x) added Groq and Mistral through litellm, OpenAI-style function tools and an onnxruntime-genai backend, and dropped the old tool-call style. The Python repo was last pushed in May 2026, while llguidance is updated daily.
The Verdict
Who Should Use Guidance?
Best For
- Teams running local models that need guaranteed formats
- Grammar-heavy outputs such as code, DSLs or strict JSON
- Engine builders who embed llguidance
Not Ideal For
- Apps that only call hosted APIs (constraints are weaker there)
- Teams that want a stable API with few breaking changes
What's Great
- Regex, grammar and JSON-schema constraints in plain Python
- llguidance engine is widely adopted across inference servers
- Fast: low per-token overhead, per the project
Watch Out For
- Breaking changes: 0.3.0 removed the old tool-call style
- Python repo less active than the engine (last push May 2026)
- Hundreds of open issues and pull requests
Pricing
View all features & details
Key Features
- select, regex, grammar and JSON-schema constraints
- Interleaved generation and Python control flow
- Backends: Transformers, llama.cpp, onnxruntime-genai, OpenAI and others
- OpenAI-style function tools (0.3.0)
- Monitor-guided inference (0.3.1)
- Current releases need Python 3.10 or later
llguidance Engine
- Rust, MIT, 1.0 since June 2025
- Used by OpenAI structured outputs, llama.cpp, vLLM, SGLang, Chromium
- Project claims about 1.5 ms per token for a typical JSON schema
Real-World Usage
Engine Adoption
- llguidance powers structured output in several inference engines
- OpenAI adoption dated to May 2025 by the project
How It Compares
| Feature | Guidance | Outlines | Instructor |
|---|---|---|---|
| Approach | Constrained decoding plus control flow | Constrained decoding | Validate and retry |
| Grammars and regex | Yes | Yes | No |
| Hosted APIs | OpenAI, plus Groq and Mistral via litellm | Needs local decoding | Yes, 15+ providers |
| License | MIT | See vendor | MIT |
Sources: Guidance GitHub · Releases · llguidance · llguidance blog · PyPI · see also Outlines and Instructor
Is this your tool? Add a badge