PIGuard
Academic ACL 2025 prompt-injection classifier that reduces false positives on benign trigger-word prompts
Overview
PIGuard is the official code, dataset, and model release accompanying the ACL 2025 paper "PIGuard: Prompt Injection Guardrail via Mitigating Overdefense for Free" by Hao Li, Xiaogeng Liu, Ning Zhang, and Chaowei Xiao. It's a research-grade text-classification model (fine-tuned via a training strategy the paper calls "Mitigating Over-defense for Free," or MOF) aimed at a specific failure mode of existing prompt-guard classifiers: over-defense, where benign prompts that merely contain words common in injection attacks (e.g., "ignore," "system prompt," "instructions") get incorrectly flagged as attacks. The project was originally released under the name InjecGuard and later renamed to PIGuard for licensing reasons—the paper site and Hugging Face listings still reference the InjecGuard name in places. This is an academic single-paper release with model weights on Hugging Face and code/datasets on GitHub, not a maintained commercial product—there's no company, support contract, dashboard, or SLA behind it.
The Verdict
Who Should Use PIGuard?
Best For
- Researchers benchmarking prompt-guard models for over-defense / false-positive bias on trigger words
- ML engineers looking for a permissively licensed (MIT) classifier checkpoint to fine-tune or evaluate against
- Teams wanting the NotInject benchmark dataset to stress-test their own guardrail models for benign-prompt false positives
- Academic or hobbyist projects comfortable self-hosting a Hugging Face model with no vendor support
Not Ideal For
- Teams needing a maintained, production-supported guardrail with SLAs, uptime guarantees, or a dashboard
- Organizations that need vendor accountability, security patching cadence, or enterprise procurement/compliance paperwork
- Non-ML teams without the infrastructure to host and serve a classifier model themselves
- Anyone needing indirect-injection or agent-reasoning defenses—PIGuard is a single input classifier, not a multi-layer agent-security framework
What's Great
- Directly targets a real, measurable weakness in existing prompt-guard models: several drop to near-random (~60%) accuracy on benign prompts containing trigger words
- Introduces NotInject, a reusable 339-sample benchmark for measuring over-defense, useful even if you don't adopt the PIGuard model itself
- Peer-reviewed via ACL 2025, giving it more methodological scrutiny than a typical unreviewed GitHub project
- MIT-licensed code and freely downloadable model weights on Hugging Face
- Reports beating the prior best model by roughly 30.8% on the NotInject over-defense benchmark, per the project's own README
Watch Out For
- Single-author academic repo with no company behind it—not a commercially supported product
- Renamed mid-life from InjecGuard to PIGuard for licensing reasons, which creates some confusing/inconsistent naming across the paper site, Hugging Face, and GitHub
- Performance and over-defense-reduction numbers come from the authors' own paper and README; independent third-party validation is limited
- No dedicated product website—the "website" is a GitHub Pages paper landing page, not a maintained docs/marketing site
- Only addresses direct prompt injection input classification; no coverage of indirect injection, agent reasoning, or tool-output scanning
Pricing
View all features & details
Key Features
- MOF (Mitigating Over-defense for Free): training strategy designed to reduce bias toward flagging benign prompts that contain injection-attack trigger words
- NotInject: 339-sample benign evaluation dataset spanning three trigger-word-density tiers and four topic areas (Common Queries, Technique Queries, Virtual Creation, Multilingual Queries)
- Pretrained sequence-classification model published on Hugging Face (`leolee99/PIGuard`), usable directly via `transformers` `pipeline("text-classification", ...)`
- Evaluated across four benchmarks: NotInject, PINT, Wildguard-Benign, and BIPIA
- Full training and evaluation code released on GitHub alongside the paper
Use Cases
- Academic benchmarking of prompt-injection classifiers for both attack recall and benign-prompt false-positive rate
- Stress-testing an existing in-house or vendor guardrail against the NotInject dataset to check for over-defense
- Self-hosted, low-cost prompt-injection filtering for hobby or research projects that don't require commercial support
- Starting point for further fine-tuning on domain-specific injection/benign prompt data
How It Compares
| Feature | PIGuard | ProtectAI DeBERTa | LlamaFirewall PromptGuard |
|---|---|---|---|
| Deployment | Self-hosted, open source (Hugging Face) | Self-hosted, open source (Hugging Face) | Self-hosted, open source (part of LlamaFirewall) |
| Backing | Single-author academic project (ACL 2025 paper) | ProtectAI (commercial security vendor) | Meta (Purple Llama, used internally in production) |
| Over-defense / false-positive focus | Core contribution—purpose-built MOF training + NotInject benchmark | Not a primary design focus | Not a primary design focus |
| Benchmark dataset released | Yes—NotInject (339 samples) | No dedicated over-defense dataset | No dedicated over-defense dataset |
| License | MIT | Apache 2.0 | MIT (framework); Llama Community License (models) |
| Maturity / support | Research prototype, no commercial support | Backed by a security company, more production usage | Backed by Meta, actively maintained |
| GitHub stars | 77 | Higher (widely used base model) | 4,300+ (PurpleLlama repo) |