WhyLabs LangKit
Open-source Python toolkit for LLM prompt/response security signals, including similarity-based prompt-injection detection. Community-maintained since 2025.
Overview
WhyLabs LangKit is an open-source Python toolkit that extracts monitoring and security signals from LLM prompts and responses — text quality, sentiment, toxicity, readability, and a similarity-based prompt-injection/jailbreak detector that scores incoming prompts against a FAISS vector database of known attack strings, surfaced as a prompt.injection metric. It integrates with whylogs for statistical profiling and was originally built to feed WhyLabs' commercial LLM observability platform.
That commercial platform is no longer relevant: WhyLabs, Inc. ceased commercial operations and shut down its hosted SaaS offering in early 2025, open-sourcing its full stack (including the WhyLabs AI Control Center) under Apache 2.0 in response. LangKit itself continues to exist as a standalone open-source library, but it is now community-maintained rather than backed by an active vendor, and the GitHub repository shows no commits since November 2024. Teams evaluating it today should treat it as a free, self-hosted signal-extraction library rather than a supported product.
The Verdict
Who Should Use WhyLabs LangKit?
Best For
- Teams that already use whylogs and want lightweight, self-hosted LLM text metrics
- Prototyping similarity-based prompt-injection/jailbreak detection against a known-attack corpus
- Batch analysis of prompt/response logs for toxicity, sentiment, and readability
- Cost-sensitive projects that want a free, code-level library rather than a hosted service
Not Ideal For
- Teams needing an actively maintained, vendor-backed tool with a support SLA
- Real-time production guardrails against novel or adversarial prompt-injection attacks
- Organizations wanting a hosted dashboard or managed observability platform (WhyLabs' SaaS is discontinued)
- Anyone requiring detection techniques beyond similarity scoring against a static, aging attack database
What's Great
- Free and Apache 2.0 licensed, fully self-hostable with no vendor lock-in
- Purpose-built
prompt.injectionmetric using FAISS similarity search against known jailbreak/harmful prompts - Bundles broader LLM text metrics (toxicity, sentiment, readability, relevance) alongside the security signal
- Integrates directly with whylogs, which remains actively maintained by WhyLabs
- 35 releases and ~72 forks show meaningful historical adoption and community usage
Watch Out For
- Discontinued as a commercial product; WhyLabs ceased operations and open-sourced its platform in January 2025
- No commits or releases since November 2024 — now community-maintained, with no confirmed active roadmap
- Prompt-injection detection relies on similarity to a static, unrefreshed database of known attacks, so it will miss novel injection techniques
- No hosted dashboard or managed service remains available; the original SaaS platform's hosted access ended March 2025
- Heavy Jupyter Notebook footprint (~90% of the repo) suggests documentation/examples outpace production-hardened tooling
Pricing
View all features & details
Key Features
- Prompt-injection and jailbreak similarity scoring via FAISS against known attack prompts
- Text quality metrics: readability, complexity, grade-level scores
- Text relevance: prompt/response similarity analysis
- Sentiment and toxicity classification
- Hallucination consistency checks and refusal-pattern recognition
- Native integration with whylogs for statistical profiling and drift detection
Use Cases
- Batch or pipeline scanning of LLM prompt/response logs for security and quality signals
- Lightweight, offline prototyping of prompt-injection detection before adopting a maintained tool
- Feeding whylogs profiles into custom monitoring or alerting infrastructure
- Research and benchmarking against a known jailbreak/harmful-prompt corpus
How It Compares
| Feature | LangKit | LLM Guard | NeMo Guardrails |
|---|---|---|---|
| Maintenance Status | Community-maintained, no commits since Nov 2024 | Actively maintained | Actively maintained (NVIDIA) |
| Prompt-Injection Method | FAISS similarity vs. known attack corpus | Multiple scanners incl. ML classifiers | Programmable rails + model-based checks |
| Scope | Text metrics + security signals | Input/output security scanning suite | Full conversational guardrail framework |
| Hosted/Managed Option | None (SaaS discontinued) | None (self-hosted) | None (self-hosted) |
| License | Apache 2.0 | MIT | Apache 2.0 |
| Best Fit | Batch log analysis, whylogs users | Production input/output filtering | Real-time dialogue policy enforcement |