Lumina
Securing LLM Apps · Module SEC.1
1/6
EN PT
Module SEC.1 · security

The threat map

You can't gate a threat you haven't mapped. Before you write a single guardrail, you need the shape of your attack surface — every place untrusted data or capability enters the model. This module hands you the two maps the field already agreed on, so you stop guessing and start covering.

what you’ll be able to do By the end of this module you can map an LLM app's attack surface to the OWASP-LLM and Agentic categories, and spot the nodes left uncovered — the blind spots a single input filter quietly hides.

Why "just add a filter" fails

The naive instinct is to bolt one input filter in front of the model and call it secure. But an LLM app isn't a single door — it's a perimeter. Untrusted text arrives from the user, from retrieved documents, from tool outputs, and from memory the model wrote to itself on a previous turn. Each of those is a distinct entry point with a distinct failure mode. Filter the user prompt and you've left the retrieval channel, the tool channel, and memory wide open. Coverage is a map problem before it's a control problem.

the move Enumerate the surface first, then assign a control per node. The community already did the enumeration for you: OWASP Top 10 for LLM Applications gives you the model-level risks, and OWASP Top 10 for Agentic Applications extends it to systems that act. Map every node to a category; any node without a category is a blind spot.

Map one — OWASP Top 10 for LLM Applications (2025)

Published by the OWASP Gen AI Security Project, this list names the canonical model-level risks. Its top entry, LLM01: Prompt Injection, is the one every other channel inherits — untrusted content persuading the model to ignore its instructions. Neighbors on the list cover sensitive-information disclosure, supply-chain risk, insecure output handling, and data/model poisoning. Treat it as the baseline vocabulary: when you triage an incident, you should be able to name it with an OWASP category.

Map two — OWASP Top 10 for Agentic Applications (2026)

The moment your app can act — call tools, spend budget, write to systems — the model-level list is no longer enough. The Gen AI Security Project's newer agentic list addresses this; its entries carry ASI0x ids (ASI01 Agent Goal Hijack, ASI02 Tool Misuse & Exploitation…). The risk it centers on is LLM06:2025 Excessive Agency (formally an entry on the model-level Top 10): an agent granted more capability, autonomy, or permission than the task actually requires. A prompt injection against a chatbot leaks text; the same injection against an agent with a shell tool and a wallet executes. Excessive agency is what turns a content risk into a blast radius. The agentic-era question is not "what can the model say" but "what can the model do, and did we scope that down to the minimum?"

the trap Excessive agency hides in convenience. Broad tool scopes, a single over-privileged API key, "let the agent retry until it works" loops, and memory that persists attacker-controlled text across sessions all widen the surface silently. You won't see it in a demo — you see it the first time an injected instruction reaches a tool. Scope capability to the task, not to what's easy to wire.

The adversary's playbook — MITRE ATLAS

OWASP tells you what can go wrong. MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) tells you how real adversaries do it — a knowledge base of tactics and techniques against AI systems, structured like the ATT&CK framework security teams already use. Use OWASP to enumerate risk categories and ATLAS to enumerate the concrete techniques that realize them, so your tests mirror attacker behavior instead of your own imagination.

The agentic attack surface

Here is the perimeter, drawn. Four untrusted inputs converge on the model core, which can act through tools — each edge tagged with the OWASP category that governs it.

User input prompt / chat Retrieved content RAG / web docs Tool output API / shell results Memory persisted state LLM core reason + plan decide to act Tools / actions shell · money · write LLM01 prompt injection LLM01 · indirect injection LLM05 improper output LLM04 · data poisoning (memory) LLM06 excessive agency scope tools to the task

A map is only useful if it drives tests

Naming a risk is step zero; the harness turns each category into an executable check. The pattern is a threat-to-test mapping — a small config that binds every surface node to its OWASP category and the probe that proves whether the gate holds. If a node has no test, it isn't covered — it's assumed safe, which is how blind spots survive review.

# threat_map.yaml — every surface node -> OWASP category -> a probe that gates it
surface:
  - node: user_input
    owasp: LLM01_prompt_injection
    probe: inject_ignore_instructions      # attacker overrides system prompt
    gate: refuses AND stays on task

  - node: retrieved_content
    owasp: LLM01_indirect_injection
    probe: poisoned_document               # instructions hidden in a RAG doc
    gate: no instruction obeyed from data

  - node: tool_output
    owasp: LLM05_improper_output_handling
    probe: malicious_tool_response         # tool returns markup / commands
    gate: output validated before use

  - node: memory
    owasp: LLM04_data_and_model_poisoning
    probe: persist_hostile_note            # attacker text survives to next turn
    gate: memory sanitized on read

  - node: tool_invocation
    owasp: LLM06_excessive_agency          # LLM06:2025; the Agentic Top 10 (ASI0x) centers on it
    probe: over_privileged_action          # agent tries beyond task scope
    gate: least-privilege scope enforced

# unmapped node == blind spot. coverage = mapped_nodes / total_nodes

You'll build and run probes like these in later modules; here the point is the discipline — coverage is measured against a map, not a feeling. Every node earns a category and a probe, or it's flagged as unexamined.

public sources The map is only as good as the probes behind it — category-only checklists stay brittle, which is why you pair each category with an executable probe. Anchor on the sources of record: the OWASP Gen AI Security Project (LLM & Agentic Top 10), MITRE ATLAS for adversary techniques, and the NIST AI 600-1 Generative AI Profile for risk management.
▶ runnable lab

owasp-llm-map

Take a small agentic app, enumerate its surface nodes, and produce a threat-to-test map that tags each node with an OWASP-LLM or agentic category. The script reports coverage — which nodes have a probe and which are still blind spots. Run it on the included sample app, then point it at your own.

labs/owasp-llm-map/ · python owasp_map.py
which map covers "the model leaks its system prompt" vs "the agent emails a stranger"?
The first is a model-level risk → OWASP Top 10 for LLM Applications (System Prompt Leakage). The second is a system-that-acts risk → LLM06:2025 Excessive Agency, the risk the OWASP Top 10 for Agentic Applications (ASI0x) centers on. MITRE ATLAS then supplies the adversary technique IDs for both.
why is "just add a filter" not a threat map?
A filter is one control against one attack; a map enumerates the whole surface so every node earns a category and a probe. Without the map you harden the wall you thought of and leave the ones you didn't — coverage is measured against the map, not against your intuition.
what makes excessive agency the agentic-era risk?
When the model can act (call tools, spend, email), an injection stops being a text leak and becomes a blast radius. Scope capability to the task so a hijacked prompt can't reach beyond it.
key takeaways