Operational AI Security guide for LLMs.

It is the list of the ten most critical security risks in applications that use large language models (LLMs): prompt injection, insecure output handling, training data poisoning, model denial of service, supply-chain vulnerabilities, sensitive information disclosure, insecure plugin design, excessive agency, overreliance on the model and unbounded consumption. It is maintained by the OWASP Foundation with input from hundreds of professionals and is the most widely used operational reference in the industry to audit and harden generative-AI systems.

On the OWASP Top 10 for LLM applications. Each of the ten risks (LLM01 Prompt Injection through LLM10 Unbounded Consumption) is translated into binary checks an engineering team can apply against their own system in under one hour. Each risk has an anti-pattern observed in production and the question a real attacker would ask.

Prompt injection is the most common attack against LLMs: an attacker plants instructions inside an input that the model processes (a user message, a PDF the user uploads, an email, a web page an agent browses) so the model does things it should not: exfiltrate data from the system prompt, execute privileged tools or ignore moderation rules. It is detected by combining targeted red teaming (direct and indirect injections), multi-turn testing and monitoring of anomalous outputs. The checklist marks the minimum verifications for LLM01.

They serve different purposes. OWASP LLM Top 10 is operational and developer-oriented: "what do I have to verify in my code?". MITRE ATLAS is a map of adversarial tactics and techniques against AI systems, useful for red teamers and SOCs. NIST AI RMF is a high-level risk-management framework for AI leadership, aligned with governance and compliance. In real audits we combine all three: ATLAS for attack scenarios, OWASP for technical verifications, NIST AI RMF to fit the result into internal governance and the EU AI Act.

You simulate an authorized attacker against the full system: direct and indirect prompt injection, jailbreaks, system-prompt leakage, data exfiltration through context, privilege escalation via tools (function calling), poisoning of RAG sources and multi-turn attacks. The output is a list of reproducible findings with proof-of-concept and remediation prioritized by impact. Important: red teaming an LLM is not an automated scan; it requires human creativity and domain knowledge of the system.

The most widely used are Garak (NVIDIA's vulnerability scanner with batteries of probes for jailbreaks, leakage and toxicity), PyRIT (Microsoft's framework to automate red teaming), promptfoo (prompt and detection testing), Giskard (quality and robustness testing) and Rebuff (defense against prompt injection). None replaces a manual red team; they are useful for continuous regression and as a complement to human review.

Function calling and tools turn the LLM into a system with real-world effects: read mail, write to a database, make payments, browse websites. Every tool extends the attack surface. Typical risks: excessive agency (LLM09), confused deputy (the tool trusts the LLM more than it should), prompt-injection chaining (a malicious email triggering a wire transfer), overly broad permissions, missing human approval on sensitive steps. Minimum requirements: principle of least privilege per tool, domain allowlists, human approval on irreversible operations and an auditable log of every call.

No, RAG is not safe by default: it adds two attack surfaces on top of an isolated LLM. One, indirect prompt injection through retrieved documents: if your vector store contains a malicious PDF, its instructions reach the model as if they came from the user. Two, leaks of sensitive data when the system retrieves documents the current user should not have access to. Essential mitigations: document-level access control applied at query time (not in the prompt), sanitization of external sources before indexing, source labelling on every chunk and precedence of system-prompt instructions over those in retrieved context.

No. It also works for systems about to launch, advanced prototypes, or internal audits. The only requirement is that there is a concrete architecture to evaluate.

You receive the PDF in your inbox. If after reading it you want us to walk the checks against your specific system, we run a free diagnostic; just reply to the email.