OWASP LLM Top 10 Benchmark Mapping: What Tests Each Risk

The short answer to an OWASP LLM Top 10 benchmark mapping: only LLM01 Prompt Injection has several mature, comparable benchmarks. Six categories have only research-grade suites, and LLM03 Supply Chain, LLM05 Improper Output Handling and LLM10 Unbounded Consumption have no standard benchmark at all, so they need scanners, assertions and load tests instead.

The gap is structural. The OWASP Top 10 for LLM Applications 2025 lists application risks, while nearly every public benchmark scores a model or agent against a generic corpus. Risks that live in your architecture, such as where model output lands, who can query a vector store, or what a request costs, cannot be scored by a dataset that never sees that architecture. The practical recommendation: map at the test-case level rather than the suite level, pin the taxonomy version, and report untested categories as untested instead of leaving the cell blank.

The mapping, category by category

The table uses 2025 IDs. “Coverage” is an editorial grade based on each suite’s published scope, not a measured score. Tool-level entries come from Promptfoo’s OWASP LLM Top 10 documentation and garak’s probe source, normalized to 2025 numbering.

Four rows need a caveat before anyone pastes this into a procurement scorecard.

HarmBench and JailbreakBench sit under LLM01 because the LLM01 entry states that jailbreaking is a form of prompt injection. What those suites score, though, is whether harmful content comes out, and harmful content is not one of the ten categories. Their differences are worked through in AdvBench vs HarmBench vs JailbreakBench.

Tensor Trust’s extraction set measures LLM07 leakage, but the LLM07 entry says “the system prompt should not be considered a secret, nor should it be used as a security control.” A strong extraction score does not make it safe to put credentials or authorization logic in a prompt.

PoisonedRAG covers one of the five risks in the LLM08 entry. Nothing in the table scores embedding inversion or cross-tenant leakage in a shared vector database; those are access-control tests against your own deployment.

LLM10:2025 absorbed much of what the 2023 list called Model Theft. The entry lists model extraction via API and functional model replication alongside denial of wallet and input floods, so an “LLM10” finding in an older report may describe a different risk.

Why does LLM01 dominate benchmark coverage?

LLM01 dominates because prompt injection is the one OWASP risk a fixed corpus of inputs can exercise on any target, with no knowledge of the application behind it. That makes it cheap to benchmark and easy to compare across models, so the suites have accumulated scale, published baselines and research attention.

The depth is real. CyberSecEval 2 added prompt injection and code interpreter abuse to Meta’s suite and reported that all tested models showed between 26% and 41% successful prompt injection tests. AgentDojo moves the problem into tool-using agents with 97 user tasks and 629 security test cases, and scores utility under attack alongside attack success, which is why it also earns a partial LLM06 entry. Tensor Trust contributes over 126,000 player-written attacks and 46,000 defenses from an online game, split into hijacking (LLM01) and extraction (LLM07).

Coverage still has edges. The LLM01 entry includes a multimodal injection scenario that text-only suites do not exercise, and indirect injection through retrieved content appears in AgentDojo but not in the harmful-content suites. A detector or guardrail in front of the model is a separate measurement whose false-positive rate matters as much as its catch rate, covered in how to benchmark a prompt-injection detector honestly. For agent reporting, see measuring prompt-injection robustness in tool-using agents.

Which OWASP LLM categories have no real benchmark?

LLM03 Supply Chain, LLM05 Improper Output Handling and LLM10 Unbounded Consumption have no standard benchmark, because each is a property of the system around the model rather than of the model’s answers. They are tested with artifact scanners, sink-level injection tests and load or cost tests, and the output is a list of findings, not a comparable score.

LLM03. The useful control is static scanning of model artifacts before they load. Protect AI’s ModelScan reads H5, Pickle and SavedModel files without executing them and flags unsafe code. That yields a pass or fail per artifact, not a number you can compare across vendors, and it says nothing about dataset provenance or third-party API dependencies.

LLM05. The LLM05 entry lists remote code execution through exec or eval, XSS from generated JavaScript or Markdown, unparameterized SQL, and path traversal. In every example the vulnerable component is the downstream sink. garak’s ANSI escape probes check whether a model will emit terminal control codes, a useful precondition, but whether that output does damage depends on your log viewer or terminal. Real coverage means running LLM-shaped payloads through the sink’s own tests.

LLM10. The sponge examples paper showed crafted inputs raising energy consumption by a factor of 10 to 200 on vision and language models. That demonstrates the attack class but is not a reusable benchmark; LLM10 is tested against the deployment, not the model.

LLM04 deserves a footnote. BackdoorLLM is a genuine benchmark, covering data poisoning, weight poisoning, hidden-state manipulation and chain-of-thought hijacking across more than 200 experiments, but it evaluates backdoors you plant yourself. It cannot tell you whether a model you downloaded was poisoned, and Promptfoo’s documentation is explicit that its tooling helps detect the effects of poisoning rather than prevent it.

The 2023 vs 2025 numbering trap in scanner tags

Scanner tags written against the 2023-24 list silently mis-map under the 2025 list. Only LLM01 kept both its number and its meaning. LLM09 and LLM10 kept a related meaning, and the other seven IDs now point at different risks.

garak shows how this plays out. Its probes carry MISP-style tags such as owasp:llm01. The leakreplay probes, which test whether a model replays training documents, are tagged owasp:llm10 and owasp:llm06. Under the 2023 list that reads as Model Theft and Sensitive Information Disclosure, which fits. Under 2025 it reads as Unbounded Consumption and Excessive Agency, which does not. In the ansiescape module, AnsiEscaped carries both owasp:llm02 and owasp:llm05, which looks like an attempt to cover both lists’ output-handling IDs, while AnsiRawTokenizerHF carries only owasp:llm05: Supply Chain in 2023, Improper Output Handling in 2025. Either reading is defensible for a tokenizer check, and that is the problem.

Promptfoo’s owasp:llm:01-style shorthands follow the 2025 numbering. Merge a garak report and a Promptfoo report on the raw tag string and the resulting matrix silently combines unrelated risks.

The fix is a crosswalk keyed on tool, tool version and probe, resolving to a 2025 ID, with a MITRE ATLAS technique ID as a second key. How each harness structures its output is compared in the LLM red teaming tools benchmark.

How to build a mapping you can defend

1. Pin the taxonomy version. Write “OWASP LLM Top 10 2025” in the report header and normalize every tool tag to it before aggregating.
2. Map test cases, not suites. AgentDojo cases test LLM01 or LLM06 depending on which security property they check; a suite-level label hides that.
3. Record the target layer. A model-only run, an agent run and a run against the deployed application with guardrails in place answer different questions, and results do not transfer between them.
4. Grade coverage in three states. Measured, partially measured, not measured. A blank cell gets read as a pass.
5. Report the denominator and the cost. Every success rate needs its attempt count, judge and corpus version, and every defense needs its false-positive or utility number beside it.

The pinning discipline behind step 5 is set out in designing a reproducible AI-security eval harness, and the end-to-end method in how to benchmark LLM security. For what each public suite measures beyond the OWASP lens, see the AI and LLM security benchmarks reference.

FAQ

is there an official owasp llm top 10 benchmark

No. The OWASP Top 10 for LLM Applications publishes risk descriptions, example attack scenarios and mitigations, not a scored test corpus. Coverage comes from third-party suites such as AgentDojo, CyberSecEval and Tensor Trust, and from scanners like garak and Promptfoo that tag tests with OWASP IDs. Every mapping is someone’s interpretation and should say whose.

which tools map tests to the owasp llm top 10

Promptfoo and garak both map tests to OWASP IDs. Promptfoo ships an owasp:llm red-team preset with per-category shorthands such as owasp:llm:01 that follow the 2025 list. garak tags individual probes with strings like owasp:llm01, but some tags follow the 2023-24 numbering, so check each probe before trusting an aggregated report.

how does mitre atlas map to owasp llm top 10

Several 2025 OWASP entries name related MITRE ATLAS techniques, giving a starting crosswalk. LLM01 references AML.T0051.000 and AML.T0051.001 for direct and indirect prompt injection, plus AML.T0054 for jailbreaks. LLM07 references AML.T0051.000 for meta prompt extraction, and LLM09 references AML.T0048.002, societal harm. Carry both IDs in every report row.

how do you test llm10 unbounded consumption

Test LLM10 with load, quota and cost tests against the deployed application rather than with a benchmark. Send oversized and repetitive inputs, measure tokens and latency per request, confirm that rate limits and spend caps actually trigger, and probe for output that never terminates. Promptfoo maps its divergent-repetition plugin, which tests repetitive-pattern prompts, to this category.

Related across the network

• LLM Security Audit Checklist: 10 Controls Mapped to OWASP — aisecweekly.com
• LLM Red Teaming Tools Compared: Garak, PyRIT & More — jailbreaks.fyi