All articles
Published benchmark results for AI security tooling, collected in one place. Scores from vendor, academic and open-source evaluations of LLM scanners, prompt-injection detectors and jailbreak classifiers, each recorded with the conditions it was produced under.
-
RAID Benchmark AI Detector Results: What the Scores Mean
RAID scores AI-text detectors at a fixed 5% false-positive rate across 11 models and 11 attacks. What top scores mean and where they mislead.
-
OWASP LLM Top 10 Benchmark Mapping: What Tests Each Risk
Which public benchmarks and scanner probes test each OWASP LLM Top 10 2025 risk, where coverage is thin, and why tool tags still use 2023 IDs.
-
AI Jailbreak Benchmark Leaderboard: 2026 Results
Compare AI jailbreak benchmark results from FAR.AI, JailbreakBench, HarmBench, StrongREJECT, and MLCommons by scope, judge quality, and procurement value.
-
Prompt Injection Scanner Tools: Benchmarks & Evasion
How prompt injection scanners compare on published robustness and evasion benchmarks: cloud guardrails, Lakera Guard, Prompt Guard 2, and NeMo.
-
AI Firewall Placement: Lakera, NeMo, Bedrock, Azure
Where Lakera Guard, NeMo Guardrails, Bedrock Guardrails, and Azure AI Content Safety sit in the LLM request path, and what each placement costs.
-
10 AI and LLM Security Benchmarks Compared
Compare 10 AI security benchmarks for LLMs, agents, detectors, and software-security tasks, including their scope, metrics, limits, and reporting traps.
-
AI Security Testing: A Method for LLM and Agent Systems
AI security testing spans application, model, infrastructure, and data risks, with repeatable cases, calibrated judging, metrics, and CI checks.
-
How to Benchmark LLM Security: A Repeatable Method
Benchmark LLM security repeatably: define the threat model, pick suites that map to it, pin the target, and report attack success rate with refusal rate.
-
Open Source LLM Security Scanners Compared
Garak, NeMo Guardrails, PyRIT, and ARTKIT compared: how the leading open source LLM security scanners differ on coverage, fit, and maintenance.
-
AI Model Evaluation Metrics: Accuracy, ASR, Refusal, FPR
How AI security evaluations are scored: attack success rate, benign refusal rate, detector TPR and FPR, and judge agreement, with the math behind each.
-
40 AI Security Tools Compared: 2026 Directory
Compare 40 AI and LLM security tools by category, function, open-source status, and maturity across scanners, guardrails, detection, and monitoring.
-
AI Guardrail Tools Benchmarked: Lakera, NeMo, Bedrock
AI guardrail tools benchmarked on adversarial robustness, latency, and cost: Lakera Guard, NVIDIA NeMo, AWS Bedrock Guardrails, and Guardrails AI.
-
LLM Red Teaming Tools Benchmarked: Garak, PyRIT, Promptfoo
How Garak, PyRIT, and Promptfoo compare for reproducible red-team benchmarks, with a capability matrix and roles for HarmBench and JailbreakBench.
-
AI Agent Security Testing: A Practical Guide
Testing AI agent security needs a different approach than static LLM red teaming. The attack surface, a test methodology, and the OWASP agentic checklist.
-
Designing a Reproducible AI-Security Eval Harness
A reproducible AI-security eval is an engineering artifact: a harness that separates corpus, target, judge, and report, plus what a scanner run must pin.
-
Measuring Prompt-Injection Robustness in Tool-Using Agents
Prompt-injection robustness for an agent is not a single number — it is utility-under-attack against targeted attack success.
-
AdvBench vs HarmBench vs JailbreakBench: Key Differences
AdvBench, HarmBench, and JailbreakBench are not interchangeable, and treating them as one undermines every comparison built on top.
-
Red-Team Eval Methodology: ASR and Refusal Rate
A paired red-team evaluation measures attack success and benign refusal on separate corpora, with a calibrated judge and fixed model configuration.
-
Benchmarking LLM Jailbreak Resistance: Attack Success Rate
Attack success rate is the headline metric for jailbreak resistance, and almost everyone computes it in a way that isn't comparable across runs.
-
How to Benchmark a Prompt-Injection Detector Honestly
Most prompt-injection detector benchmarks are broken before the first request. Here is a test design that produces a number you can actually trust.
-
Choosing an LLM on Security Evidence, Not MMLU Scores
MMLU-style capability scores do not measure how a model behaves under attack. What to measure before choosing an LLM for a security-sensitive deployment.