PyRIT iconPyRIT

Microsoft's open-source Python Risk Identification Tool for automated, multi-turn red teaming of generative AI systems

4.7k+ GitHub Stars
MIT License
2024 Open-Sourced

Overview

PyRIT (Python Risk Identification Tool) is Microsoft's open automation framework for finding risks in generative AI systems. Microsoft released it in February 2024 and says its own AI Red Team has used it in more than 100 red-teaming operations. Instead of a fixed scan, PyRIT lets a red teamer script multi-turn attacks that adapt to the target's answers, score results with other models, and keep every conversation in a memory store for later analysis. The project moved from the Azure GitHub org to microsoft/PyRIT, and v1.0 (July 2026) introduced a new architecture of scenarios, techniques and registries.

The Verdict

Who Should Use PyRIT?

Best For

  • Security researchers running custom, multi-turn red-team campaigns
  • Teams that want to script attacks in Python and keep full transcripts
  • Organizations already following Microsoft's AI red-teaming practices

Not Ideal For

  • A one-command baseline scan (garak is quicker to start)
  • A deterministic CI gate or a safety benchmark score
  • Non-developers who want a point-and-click product

What's Great

  • Automates multi-turn, adaptive attacks that change tactics based on the target's replies
  • Battle-tested: Microsoft reports use in more than 100 internal red-team operations
  • Memory component keeps every prompt and response for post-exercise review
  • Free under the MIT license, actively developed (v1.1.0 in Sep 2026)

Watch Out For

  • Model-based scoring is non-deterministic and can flag false positives
  • Not a replacement for manual red teaming, and not a CI gate
  • v1.0 and v1.1 shipped breaking changes; older tutorials may not work
  • Run it in an isolated environment: it stores credentials and full transcripts

Pricing

View all features & details

Key Features

  • Multi-turn attack orchestration with adaptive strategies
  • Pluggable targets and scorers, including LLM-based scoring
  • Memory store for analysis after an exercise
  • Scenario catalog and GUI (v1.1)

Recent Releases

  • v1.1.0 (Sep 2026): undetermined-score support, more scanner attacks
  • v1.0.1 (Jul 2026): structured OpenAI refusals counted as refusals
  • v1.0.0 (Jul 2026): new scenarios, techniques, registries and memory design

Use Cases

  • Red-teaming a copilot or agent before launch
  • Testing jailbreak and harmful-content resistance over long conversations
  • Building repeatable internal red-team playbooks

How It Compares

Feature PyRIT garak Promptfoo Mindgard
Maker Microsoft NVIDIA Promptfoo Mindgard
License MIT Apache-2.0 MIT Proprietary
Attack style Adaptive multi-turn Probe batteries Config-driven test suites Recon plus adaptive attacks
Interface Python library, GUI CLI CLI and library Hosted platform
Price Free Free Free (OSS) Contact sales

Is this your tool? Add a badge