garak icongarak

NVIDIA's open-source LLM vulnerability scanner that probes models for jailbreaks, prompt injection, data leakage, toxicity and hallucination

9.9k+ GitHub Stars
Apache-2.0 License
v0.17.1 Latest Release (Oct 2026)

Overview

garak (Generative AI Red-teaming and Assessment Kit) is an LLM vulnerability scanner maintained by NVIDIA. It runs batteries of probes against a model or chatbot (jailbreaks, prompt injection, data leakage, misinformation, toxicity, hallucination), scores the answers with detectors, and writes JSONL reports plus a hit log. It talks to Hugging Face, OpenAI, AWS Bedrock, Cohere, Groq, NVIDIA NIM, Replicate and any REST endpoint, so security teams can run the same scan on many targets before release. The design is described in an arXiv paper (2406.11036, June 2024).

The Verdict

Who Should Use garak?

Best For

  • Security and ML teams that want a free baseline scan before shipping an LLM app
  • Comparing several models or providers with the same probe set
  • Teams comfortable with a Python CLI and writing their own probe plugins

Not Ideal For

  • Buyers who want a managed dashboard, SLAs and vendor support
  • Deep multi-turn, scenario-style red-team campaigns (PyRIT fits better)
  • Teams without time to review flagged hits by hand

What's Great

  • Broad probe coverage across many failure types, from jailbreaks to data leakage
  • Frequent releases; an independent 2024 comparison rated it strong on attack effectiveness
  • Plugin architecture: probes, detectors, generators, harnesses and evaluators are all swappable
  • Free under Apache-2.0 and runs entirely on your own machine

Watch Out For

  • Keyword and classifier detectors produce false positives, so hits need human review
  • CLI-first: no hosted UI or team workflow
  • Release notes include breaking fixes to probes and detectors, so pin versions in CI

Pricing

View all features & details

Key Features

  • Static, dynamic and adaptive probes
  • Probe families for jailbreaks, prompt injection, leakage, toxicity, misinformation and hallucination
  • Pick probes by spec and browse the catalog from the CLI
  • JSONL report plus a hit log of every failure
  • Generators for Hugging Face, OpenAI, Bedrock, Cohere, Groq, NIM, Replicate and REST

Recent Releases

  • v0.17.1 (Oct 2026): hit@k attack-success metric, Python 3.14 support
  • v0.17.0 (Sep 2026): probe and detector fixes
  • v0.16.0 (Aug 2026): IntentProbe and a native Anthropic generator

Use Cases

  • Pre-release security scan of a chatbot or RAG app
  • Regression scan after a model or prompt change
  • Comparing jailbreak resistance across model vendors

How It Compares

Feature garak PyRIT Promptfoo Mindgard
Maker NVIDIA Microsoft Promptfoo Mindgard
License Apache-2.0 MIT MIT Proprietary
Approach Probe-and-detect scanner Multi-turn attack framework Eval and red-team CLI Managed testing platform
Price Free Free Free (OSS) Contact sales
Best for Baseline scans Custom red-team operations CI checks on prompts Continuous enterprise testing

Is this your tool? Add a badge