Guidance iconGuidance

MIT library for constrained LLM generation (regex, grammars, JSON schema), powered by llguidance, the engine behind structured outputs in llama.cpp and vLLM

21.8k+ GitHub Stars
~1.5 ms Mask per Token (llguidance)
MIT License

Overview

Guidance is "a guidance language for controlling large language models", first released by Microsoft and now maintained in the guidance-ai GitHub organization. Instead of prompting and hoping, you write Python programs that mix free generation with hard constraints: select from options, match a regex, follow a context-free grammar or a JSON schema. Where the backend exposes the model's tokens (local models such as Transformers and llama.cpp), the constraints are enforced token by token, so the output fits the format. Its engine, llguidance, is a separate Rust library that reached 1.0 in June 2025; the project lists OpenAI's JSON-schema structured outputs, llama.cpp, vLLM, SGLang, Chromium, mistral.rs and onnxruntime-genai as users, and claims about 1.5 ms of mask computation per token for a typical JSON schema. Recent releases (0.3.x) added Groq and Mistral through litellm, OpenAI-style function tools and an onnxruntime-genai backend, and dropped the old tool-call style. The Python repo was last pushed in May 2026, while llguidance is updated daily.

The Verdict

Who Should Use Guidance?

Best For

  • Teams running local models that need guaranteed formats
  • Grammar-heavy outputs such as code, DSLs or strict JSON
  • Engine builders who embed llguidance

Not Ideal For

  • Apps that only call hosted APIs (constraints are weaker there)
  • Teams that want a stable API with few breaking changes

What's Great

  • Regex, grammar and JSON-schema constraints in plain Python
  • llguidance engine is widely adopted across inference servers
  • Fast: low per-token overhead, per the project

Watch Out For

  • Breaking changes: 0.3.0 removed the old tool-call style
  • Python repo less active than the engine (last push May 2026)
  • Hundreds of open issues and pull requests

Pricing

View all features & details

Key Features

  • select, regex, grammar and JSON-schema constraints
  • Interleaved generation and Python control flow
  • Backends: Transformers, llama.cpp, onnxruntime-genai, OpenAI and others
  • OpenAI-style function tools (0.3.0)
  • Monitor-guided inference (0.3.1)
  • Current releases need Python 3.10 or later

llguidance Engine

  • Rust, MIT, 1.0 since June 2025
  • Used by OpenAI structured outputs, llama.cpp, vLLM, SGLang, Chromium
  • Project claims about 1.5 ms per token for a typical JSON schema

Real-World Usage

Engine Adoption

  • llguidance powers structured output in several inference engines
  • OpenAI adoption dated to May 2025 by the project
llguidance README · Guidance blog (self-reported)

Community

  • About 1.2k forks of the Python library
  • llguidance repo pushed Nov 2026

How It Compares

Feature Guidance Outlines Instructor
Approach Constrained decoding plus control flow Constrained decoding Validate and retry
Grammars and regex Yes Yes No
Hosted APIs OpenAI, plus Groq and Mistral via litellm Needs local decoding Yes, 15+ providers
License MIT See vendor MIT

Sources: Guidance GitHub · Releases · llguidance · llguidance blog · PyPI · see also Outlines and Instructor

Is this your tool? Add a badge