10 AI and LLM Security Benchmarks Compared

An LLM security benchmark is a fixed test corpus plus a scoring rule that measures one security property of a model or agent. The ones to know: HarmBench and JailbreakBench for jailbreak resistance, AgentDojo and Agent Security Bench (ASB) for agent hijacking, CyberSecEval for cyber capability and insecure code, and PINT for injection-detector accuracy.

“AI security benchmark” is a label stretched across at least four different measurement problems. A suite that scores how often a model produces harmful text answers nothing about whether an agent can be hijacked through a calendar invite, and a suite that scores agent hijacking answers nothing about whether a detector flags too much benign traffic. Picking the wrong one produces a number that is technically correct and completely irrelevant to the risk you were asked about.

The ten public suites below come up most often, each with what it measures and the question it cannot answer. Three further agent suites, Agent Security Bench, InjecAgent and b3, are covered in the agent section.

LLM security benchmarks at a glance

Scale figures are as reported by each project at publication. Several of these are living repositories and have grown since.

Jailbreak and refusal suites

These measure the model, in isolation, with no tools attached. They answer “will it say the thing”, not “will it do the thing”.

AdvBench originated with the greedy coordinate gradient attack work on universal adversarial suffixes. Its 520 harmful behaviours and 574 harmful strings became the default corpus for optimisation-based attacks, and its ubiquity is now also its weakness: it is thoroughly present in training data and in every subsequent paper’s baseline, so a low success rate against it can reflect memorisation as easily as robustness.

HarmBench was built to fix the evaluation half of the problem rather than the corpus half. It separates standard, contextual, copyright, and multimodal behaviours, and ships a trained classifier so that two teams grading the same completions land on the same verdict. If you care about comparing attacks against each other, the standardised judge matters more than the behaviour count.

Behind a quoted HarmBench number sit 510 behaviours, 400 textual and 110 multimodal, across seven semantic categories from cybercrime and chemical or biological weapons to misinformation, harassment and copyright, split 100 for validation and 410 for test. Measured against human labels, the released Llama-2-13B classifier agreed 93.19 percent of the time on average, a GPT-4 judge 88.37 percent, and the classifier AdvBench used 69.93 percent. The MIT-licensed repository drives every stage through ./scripts/run_pipeline.py, taking --methods, --models, --step and a slurm, local or local_parallel mode.

JailbreakBench goes the other direction and deliberately keeps the behaviour set small, at 100 curated items drawn partly from AdvBench and the earlier Trojan Detection Challenge set and partly original. The value is the surrounding infrastructure: versioned attack artefacts, defence baselines, and a leaderboard, so a claimed improvement can be checked rather than taken on trust.

The distinctions between these three are load-bearing and routinely ignored, which is why results built on them so often fail to line up. That comparison is worked through in AdvBench, HarmBench, and JailbreakBench compared. How these sets show up in public rankings, and what those rankings can support, is covered in the current jailbreak leaderboard results.

AI agent security benchmarks

Once the model can call tools, the harmful-text question stops being the interesting one. AgentDojo, AgentHarm and Agent-SafetyBench split the agent problem into different halves, and none of them substitutes for another.

AgentDojo frames agent security as prompt injection against tool-using agents in realistic environments such as an email client, a banking site, and a travel booking flow. Its structure is the important part: it scores utility under attack alongside targeted attack success, so a defence that blocks every injection by crippling the agent scores badly rather than perfectly. That two-axis framing is the correct way to report agent robustness, and it is expanded on in measuring prompt-injection robustness in tool-using agents.

AgentHarm asks a different question: given an explicitly harmful multi-step task, does the agent carry it out. Its 110 base behaviours (440 with augmentations, per the AgentHarm paper) span 11 harm categories including fraud, cybercrime, disinformation, and harassment, and it reports refusal behaviour alongside completion, which keeps a model that refuses from being scored the same as a model that tries and fails.

Agent-SafetyBench is the broadest of the three, with 349 interaction environments and 2,000 test cases covering 8 risk categories and 10 recurring failure modes. Its contribution is coverage of unsafe behaviour that nobody attacked into existence: an agent that deletes the wrong records or leaks a credential through ordinary tool use, with no adversary in the loop at all. Assessments that only run adversarial suites systematically miss this class.

Agent Security Bench (ASB)

Agent Security Bench (ASB), accepted at ICLR 2025, attacks LLM agents at every stage they expose: system prompt, user prompt, tool output and memory retrieval. The ASB paper spans 10 scenarios, 10 agents, over 400 tools and 13 LLM backbones, with 27 methods: 10 prompt-injection attacks, a memory-poisoning attack, a Plan-of-Thought (PoT) backdoor, 4 mixed attacks and 11 defences. The per-attack averages in the repository show where agents break: direct prompt injection 72.68 percent, injection through tool observations 27.55 percent, memory poisoning 7.92 percent, the PoT backdoor 42.12 percent, and mixed attacks 84.30 percent, the paper’s highest.

That per-stage split is what AgentDojo does not give you. AgentDojo says whether realistic tasks survive injected content; ASB says which entry point needs the control. Each attack family runs from a YAML config on Python 3.11, for example python scripts/agent_attack.py --cfg_path config/DPI.yml. ASB is not Agent-SafetyBench, which has no attacker, and not AgentBench, which measures capability.

InjecAgent and b3

InjecAgent (ACL Findings 2024) is the quick screen: 1,054 indirect-injection cases built from 17 user tools and 62 attacker tools, scored separately for direct harm and data exfiltration.

b3, from the Breaking Agent Backbones paper, tests the backbone model rather than the agent. Its authors crowdsourced 194,331 attacks through the Gandalf Agent Breaker game, kept 210 (seven for each of 30 threat snapshots: 10 applications at three defence levels) and ran them against 34 LLMs, finding that stronger reasoning improved security while model size did not correlate with it. The full text notes that the strongest attacks are withheld: the average public attack scores 0.18 against 0.56 for the benchmark’s 210, so a run on public data understates the paper’s results. It runs through Inspect Evals after pip install inspect-evals[b3].

Agent benchmarks side by side

For the surrounding methodology, the guide to testing AI agent security covers how these suites fit into a scoped assessment.

Security engineering benchmarks

These score the model as a security practitioner rather than as a target.

SEC-bench is the one people are usually looking for when they search the name. Published in 2025 and appearing at NeurIPS, it is an automated framework that builds reproducible artefacts from real-world software vulnerabilities, using a multi-agent scaffold to construct repositories with build harnesses, reproduce the vulnerability in an isolated environment, and generate a gold patch. It then scores agents on two tasks: proof-of-concept generation and vulnerability patching. The headline figures are the useful part: the paper reports best-in-class success rates of 18.0 percent on PoC generation and 34.0 percent on patching, at a construction cost the authors put at $0.87 per instance. The gap between those two numbers is the finding worth carrying around, because patching a known, localised defect is a far easier task than demonstrating one from scratch.

Two naming collisions make this suite easy to cite wrongly. SecBench (arXiv:2412.20787), one hyphen away, is not an agent benchmark at all: it is 44,823 multiple-choice and 3,087 short-answer cybersecurity questions in Chinese and English, measuring what a model knows rather than what an agent can do. And SEC-bench Pro (arXiv:2605.26548) is the long-horizon successor to SEC-bench proper, covering 344 validated vulnerabilities in V8, SpiderMonkey, and the Linux kernel, with a top reported proof-of-concept reproduction rate of 58 percent. That figure is not comparable to SEC-bench’s 18.0 percent: different corpus, different task definition. Three papers, three unrelated numbers, near-identical names. Cite the arXiv identifier, not the name.

CyberSecEval, from Meta’s Purple Llama project, is the longest-running suite in this family. Its first three versions shipped as papers between December 2023 and August 2024: the first measured insecure code generation and helpfulness to cyberattackers; later releases added textual and visual prompt-injection evaluations, code-interpreter abuse, and offensive-capability assessments covering automated social engineering and autonomous operations. CyberSecEval 4 shifted toward defence. Meta introduced AutoPatchBench as part of it on 29 April 2025, with 136 fuzzing-found C/C++ vulnerabilities from the ARVO dataset (113 in a Lite subset), and the repository adds CyberSOCEval, built with CrowdStrike, for malware analysis and threat-intelligence reasoning. Every module runs from python3 -m CybersecurityBenchmarks.benchmark.run --benchmark=<name>. It is the right suite when the question is “does this model make attackers more effective”, and the wrong one when the question is “is my application safe”.

Detector benchmarks

Both of these evaluate a classifier rather than a generator, which means the false-positive rate is not optional.

PINT, Lakera’s Prompt Injection Test, exists because vendor detection claims were unfalsifiable. Its corpus of 4,314 inputs, split 3,016 English and 1,298 non-English, deliberately includes benign prompts and hard negatives so that a detector cannot score well by flagging everything. The multilingual share matters: detectors trained on English injection patterns commonly collapse on translated payloads, and a single-language benchmark hides that entirely.

RAID targets machine-generated-text detection, published at ACL 2024 with over 6 million generations spanning 11 models, 8 domains, 11 adversarial attacks, and 4 decoding strategies. Its contribution is the adversarial axis: detectors that look strong on clean text degrade sharply under paraphrase, homoglyph substitution, and whitespace perturbation. If a detector’s published accuracy has no adversarial condition attached, RAID is the benchmark that will embarrass it.

Building a corpus for a detector evaluation of your own, rather than consuming one of these, is covered in how to benchmark a prompt-injection detector honestly.

Choosing by the question you were actually asked

The comparability trap

Numbers from these suites travel badly. Four things break comparison, and all four are routinely omitted from published results.

• Corpus version. Every one of these repositories has changed since its paper. A score against an unnamed revision cannot be reproduced.
• The judge. HarmBench’s classifier, an LLM-as-judge, and a refusal-keyword matcher will assign three different success rates to the same completions. The judge is part of the measurement, not a detail; how each rate is defined is set out in AI security evaluation metrics.
• Attempt count. A rate computed over one attempt per behaviour and a rate computed over twenty-five are different quantities wearing the same label. Both belong in the report.
• Target configuration. System prompt, temperature, and any guardrail sitting in front of the model change the result more than most attack refinements do. The pinning discipline that fixes this is set out in reproducible AI-security eval harness, and the arithmetic of a comparable success rate in benchmarking jailbreak resistance.

What none of them measure

No suite here evaluates your system. They evaluate models and agents against generic corpora, which is exactly what makes them comparable and exactly what makes them incomplete. None of them knows your tool schema, your tenant boundaries, your document formats, or your business rules, and those are where the findings with real consequences tend to sit.

The correct use is as a baseline layer: run a public suite for comparability, then run a bespoke corpus built from your own trust boundaries for relevance, and report the two separately rather than averaging them into a single score. How to construct that second half is covered in AI security testing: a method for LLM and agent systems, and why capability scores cannot stand in for this testing when choosing a model is in choosing an LLM on security evidence, not MMLU scores.

For tooling that runs these corpora rather than defining them, see the open source LLM security scanner field guide and the 2026 red teaming tool comparison. The run-and-report method that wraps any of these suites is in how to benchmark LLM security.

FAQ

is there a single standard llm security benchmark

No. Each public suite targets a different risk, so a single “LLM security score” usually blends incompatible numbers. HarmBench or JailbreakBench covers jailbreak resistance, AgentDojo or ASB covers agent hijacking, CyberSecEval covers cyber capability, and PINT covers detector accuracy. Report each against its own question, with the corpus version and judge named.

which benchmark tests ai agents for prompt injection

AgentDojo is the default: 97 user tasks and 629 security cases scored on utility under attack and targeted attack success. Add Agent Security Bench to find which entry point (user prompt, tool output or memory) is weakest, InjecAgent for a quick 1,054-case indirect-injection screen, and b3 to compare backbone models before building an agent.

is agent security bench the same as agentbench

No. Agent Security Bench (ASB, arXiv:2410.02644) attacks agents through system prompts, user prompts, tool output and memory, and scores attack success. AgentBench (arXiv:2308.03688) is a capability benchmark across 8 environments that measures how well a model acts as an agent. Agent-SafetyBench, a third name, measures unsafe behaviour without an attacker.

how does harmbench calculate attack success rate

HarmBench’s attack success rate is the fraction of test cases where its classifier judges that the completion exhibits the target behaviour. Completions use greedy decoding and are standardised to 512 tokens, so a rate computed from sampled or longer generations, or graded by another judge, is not comparable to published HarmBench numbers.

are llm security benchmarks free to use

Mostly, yes. HarmBench, JailbreakBench, AgentDojo and ASB publish their code under the MIT licence, and CyberSecEval and b3 are publicly downloadable, so the real cost is model API calls or GPU time. The catch is attack strength: b3 withholds its most effective attacks, so its public dataset is weaker than the benchmark itself.