OWASP Top 10 for

LLM Applications
VERSION 1.1
Published: October 16, 2023
 | OWASP Top 10 for LLM Applications v1.1

OWASP Top 10 for LLM Applications

LLM02 LLM03 LLM04 LLM05
LLM01

Prompt Injection Insecure Output

Training Data
Model Denial of
Supply Chain
Handling Poisoning Service Vulnerabilities
This manipulates a large language
model (LLM) through crafty inputs, This vulnerability occurs when an LLM This occurs when LLM training data is Attackers cause resource-heavy LLM application lifecycle can be
causing unintended actions by the LLM. output is accepted without scrutiny, tampered, introducing vulnerabilities or operations on LLMs, leading to service compromised by vulnerable
Direct injections overwrite system exposing backend systems. Misuse biases that compromise security, degradation or high costs. The components or services, leading to
prompts, while indirect ones manipulate may lead to severe consequences like effectiveness, or ethical behavior. vulnerability is magnified due to the security attacks. Using third-party
inputs from external sources. XSS, CSRF, SSRF, privilege escalation, or Sources include Common Crawl, resource-intensive nature of LLMs and datasets, pre- trained models, and
remote code execution. WebText, OpenWebText, & books. unpredictability of user inputs. plugins can add vulnerabilities.

LLM06 LLM07 LLM08 LLM09 LLM10

Sensitive Information Insecure Plugin

Excessive Agency Overreliance Model Theft
Disclosure Design
LLM-based systems may undertake Systems or people overly depending on This involves unauthorized access,
LLMs may inadvertently reveal LLM plugins can have insecure inputs actions leading to unintended LLMs without oversight may face copying, or exfiltration of proprietary
confidential data in its responses, and insufficient access control. This consequences. The issue arises from misinformation, miscommunication, LLM models. The impact includes
leading to unauthorized data access, lack of application control makes them excessive functionality, permissions, or legal issues, and security vulnerabilities economic losses, compromised
privacy violations, and security easier to exploit and can result in autonomy granted to the LLM-based due to incorrect or inappropriate content competitive advantage, and potential
breaches. It’s crucial to implement data consequences like remote code systems. generated by LLMs. access to sensitive information.
sanitization and strict user policies to execution.
mitigate this.
 | OWASP Top 10 for LLM Applications v1.1

Data Flow Diagram

The diagram here presents a high level
architecture for a hypothetical large
language model application.

Overlaid in the diagram are highlighted

areas of risk illustrating how the OWASP
Top 10 for LLM Applications entries
intersect with the application flow.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM01
Direct Prompt Injection: Malicious user injects prompts to extract
sensitive information

Prompt Injection Indirect Prompt Injection: Users request sensitive data via webpage
prompts
Scam Through Plugins: Websites exploit plugins for scams.
Attackers can manipulate LLMs through
PREVENTION
crafted inputs, causing it to execute the Privilege Control: Limit LLM access and apply role-based permissions
attacker's intentions. This can be done Human Approval: Require user consent for privileged actions

directly by adversarially prompting the Segregate Content: Separate untrusted content from user prompts
Trust Boundaries: Treat LLM as untrusted and visually highlight unreliable
system prompt or indirectly through responses.

manipulated external inputs, potentially ATTACK SCENARIOS

leading to data exfiltration, social Chatbot Remote Execution: Injection leads to unauthorized access via
engineering, and other issues. chatbot
Email Deletion: Indirect injection causes email deletion
Exfiltration via Image: Webpage prompts exfiltrate private data
Misleading Resume: LLM incorrectly endorses a candidate
Prompt Replay: Attacker replays system prompts for potential further
attacks.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM02
Remote Code Execution: LLM output executed in system shell,
leading to code execution

Insecure Output Cross-Site Scripting (XSS): LLM-generated JavaScript or Markdown

causes browser interpretation.

Handling PREVENTION

Zero-Trust Approach: Treat LLM output like user input; validate and
Insecure Output Handling is a vulnerability sanitize it properly
OWASP ASVS Guidelines: Follow OWASP's standards for input validation
that arises when a downstream component and sanitization
blindly accepts large language model (LLM) Output Encoding: Encode LLM output to prevent code execution in
JavaScript or Markdown.
output without proper scrutiny. This can
lead to XSS and CSRF in web browsers as ATTACK SCENARIOS

well as SSRF, privilege escalation, or remote Chatbot Shutdown: LLM output shuts down a plugin due to a lack of
validation
code execution on backend systems. Sensitive Data Capture: LLM captures and sends sensitive data to an
attacker-controlled server
Database Table Deletion: LLM crafts a destructive SQL query, potentially
deleting all tables
XSS Exploitation: LLM returns unsanitized JavaScript payload, leading to
XSS on the victim's browser.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM03
Malicious Data Injection: Injecting falsified data during model
training

Training Data Biased Training Outputs: Model reflects inaccuracies from tainted
data

Poisoning
Content Injection: Malicious actors inject biased content into
training.

PREVENTION
Training Data Poisoning refers to
Supply Chain Verification: Verify external data sources and maintain "ML-
manipulating the data or fine-tuning process BOM" records
to introduce vulnerabilities, backdoors or Legitimacy Verification: Ensure data legitimacy throughout training
stages
biases that could compromise the model’s Use-Case Specific Training: Create separate models for different use-
security, effectiveness or ethical behavior. cases.

This risks performance degradation, ATTACK SCENARIOS

downstream software exploitation and Misleading Outputs: LLM generates content that promotes bias or hate

reputational damage. Toxic Data Injection: Malicious users manipulate the model with biased
data
Malicious Document Injection: Competitors insert false data during
model training.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM04
High-Volume Queuing: Attackers overload LLM with resource-
intensive tasks

Model Denial of Resource-Consuming Queries: Unusual queries strain system

resources

Service
Continuous Input Overflow: Flooding LLM with excessive input
Repetitive Long Inputs: Repeated long queries exhaust resources
Recursive Context Expansion: Attackers exploit recursive behavior.

Model Denial of Service occurs when an PREVENTION

attacker interacts with a Large Language Input Validation: Implement input validation and content filtering
Model (LLM) in a way that consumes an Resource Caps: Limit resource use per request
API Rate Limits: Enforce rate limits for users or IP addresses
exceptionally high amount of resources. Queue Management: Control queued and total actions
This can result in a decline in the quality of Resource Monitoring: Continuously monitor resource usage.

service for them and other users, as well as ATTACK SCENARIOS

potentially incurring high resource costs. Resource Overuse: Attacker overloads a hosted model, impacting other
users
Webpage Request Amplification: LLM tool consumes excessive
resources due to unexpected content
Input Flood: Overwhelm LLM with excessive input, causing slowdown
Sequential Input Drain: Attacker exhausts context window with sequential
inputs.
 | OWASP Top 10 for LLM Applications v1.1

EXA M PLES
LLM05
Package Vulnerabilities: Using outdated components

Vulnerable Models: Risky pre-trained models for fine-tuning

Poisoned Data: Tainted crowd-sourced data

Supply Chain Outdated Models: Using unmaintained models

Unclear Terms: Data misuse due to unclear terms.

Vulnerabilities
PREVENTI ON

Supply chain vulnerabilities in LLMs can Supplier Evaluation: Vet suppliers and policies

Plugin Testing: Use tested, trusted plugins

compromise training data, ML models, and OWASP A06: Mitigate outdated component risks

deployment platforms, causing biased Inventory Management: Maintain an up-to-date inventory

Security Measures: Sign models and code, apply anomaly detection, and
results, security breaches, or total system monitor.

failures. Such vulnerabilities can stem from

AT TACK SCEN A RI OS
outdated software, susceptible pre-trained
Library Exploitation: Exploiting vulnerable Python libraries

models, poisoned training data, and Scamming Plugin: Deploying a plugin for scams

Package Registry Attack: Tricking developers with a compromised

insecure plugin designs.
package

Misinformation Backdoor: Poisoning models for fake news

Data Poisoning: Poisoning datasets during fine-tuning.

 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM06
Incomplete Filtering: LLM responses may contain sensitive data
Overfitting: LLMs memorize sensitive data during training

Sensitive Information Unintended Disclosure: Data leaks due to misinterpretation or lack of

scrubbing.

Disclosure PREVENTION
Data Sanitization: Use scrubbing to prevent user data in training
LLM applications can inadvertently disclose Input Validation: Filter malicious inputs to avoid model poisoning
Fine-Tuning Caution: Be careful with sensitive data in model fine-tuning
sensitive information, proprietary Data Access Control: Limit external data source access.
algorithms, or confidential data, leading to
ATTACK SCENARIOS
unauthorized access, intellectual property
Unintentional Exposure: User A exposed to other user data
theft, and privacy breaches. To mitigate Filter Bypass: User A extracts PII by bypassing filters
these risks, LLM applications should Training Data Leak: Personal data leaks during training.

employ data sanitization, implement

appropriate usage policies, and restrict the
types of data returned by the LLM.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM07
Single Field Parameters: Plugins lack parameter separation
Configuration Strings: Configurations can override settings
Authentication Issues: Lack of specific plugin authorization
Insecure Plugin Raw SQL or Code: Unsafe acceptance of code or SQL.

Design PREVENTION

Parameter Control: Enforce type checks and use a validation layer

Plugins can be prone to malicious requests OWASP Guidance: Apply ASVS recommendations
Thorough Testing: Inspect and test with SAST, DAST, IAST
leading to harmful consequences like data Least-Privilege: Follow ASVS Access Control Guidelines
exfiltration, remote code execution, and Auth Identities: Use OAuth2 and API Keys for custom authorization
User Confirmation: Require manual authorization for sensitive actions.
privilege escalation due to insufficient
access controls and improper input ATTACK SCENARIOS

validation. Developers must follow robust URL Manipulation: Attackers inject content via manipulated URLs
Reconnaissance and Exploitation: Exploiting lack of validation for code
security measures to prevent exploitation, execution and data theft
like strict parameterized inputs and secure Unauthorized Access: Accessing unauthorized data through parameter
manipulation
access control guidelines. Repository Takeover: Exploiting insecure code management plugin for
repository takeover.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM08
Excessive Functionality: LLM agents have unnecessary functions, risking
misuse

Excessive Agency Excessive Permissions: Plugins may have excessive access to systems
Excessive Autonomy: LLMs lack human verification for high-impact
actions.

Excessive Agency in LLM-based systems is

PREVENTION
a vulnerability caused by over-functionality,
Limit Plugin Functions: Allow only essential functions for LLM agents
excessive permissions, or too much Plugin Scope Control: Restrict functions within LLM plugins
autonomy. To prevent this, developers need Granular Functionality: Avoid open-ended functions; use specific plugins
Permissions Control: Limit permissions to the minimum required
to limit plugin functionality, permissions, User Authentication: Ensure actions are in the user's context
and autonomy to what's absolutely Human-in-the-Loop: Require human approval for actions
Downstream Authorization: Implement authorization in downstream
necessary, track user authorization, require systems.

human approval for all actions, and ATTACK SCENARIOS

implement authorization in downstream An LLM-based personal assistant app with excessive permissions and
autonomy is tricked by a malicious email into sending spam. This could be
systems. prevented by limiting functionality, permissions, requiring user approval, or
implementing rate limiting.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM09
Misleading Info: LLMs can provide misleading info without validation
Insecure Code: LLMs may suggest insecure code in software.

Overreliance PREVENTION
Monitor and Validate: Regularly review LLM outputs with consistency
Overreliance on LLMs can lead to serious checks

consequences such as misinformation, Cross-Check: Verify LLM output with trusted sources
Fine-Tuning: Enhance LLM quality with task-specific fine-tuning
legal issues, and security vulnerabilities.
Auto Validation: Implement systems to verify output against known facts

It occurs when an LLM is trusted to make Task Segmentation: Divide complex tasks to reduce risks
Risk Communication: Communicate LLM limitations
critical decisions or generate content User-Friendly Interfaces: Create interfaces with content filters and
warnings
without adequate oversight or validation. Secure Coding: Establish guidelines to prevent vulnerabilities.

ATTACK SCENARIOS
Disinfo Spread: Malicious actors exploit LLM-reliant news organizations
Plagiarism: Unintentional plagiarism leads to copyright issues
Insecure Software: LLM suggestions introduce security vulnerabilities
Malicious Package: LLM suggests a non-existent code library.
 | OWASP Top 10 for LLM Applications v1.1

EXAMPLES
LLM10
Vulnerability Exploitation: Unauthorized access due to security flaws
Central Model Registry: Centralized security for governance

Model Theft Insider Threat: Risk of employee model leaks

Side-Channel Attack: Extraction of model details through side
techniques.

LLM model theft involves unauthorized

PREVENTION & MITIGATION
access to and exfiltration of LLM models, Access Control and Authentication: Strong access controls and
risking economic loss, reputation damage, authentication
Network Restrictions: Limit LLM access to resources and APIs
and unauthorized access to sensitive data. Monitoring and Auditing: Regular monitoring of access logs
Robust security measures are essential to MLOps Automation: Secure deployment with approval workflows.

protect these models. ATTACK SCENARIOS

Model Theft: Unauthorized access and use for competition

Employee Leak: Exposure increases risks
Shadow Model Creation: Replicating models with queries
Side-Channel Attack: Extraction through side techniques.
 | OWASP Top 10 for LLM Applications v1.1

Key Reference Links

Prompt Injection attack against LLM-integrated Applications: Cornell Universit Tay Poisoning: MITR
Defending ChatGPT against Jailbreak Attack via Self-Reminder: Research Squar Backdoor Attacks on Language Models: Can We Trust Our Model’s Weights?: Mediu
OpenAI Chat Markup Language: GitHu Poisoning Language Models During Instruction Tuning: Cornell Universit
Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with ChatGPT Data Breach Confirmed as Security Firm Warns of Vulnerable Component
Indirect Prompt Injection: Cornell Universit Exploitation: Security Wee
Threat Modeling LLM Applications: AI Villag What Happens When an AI Company Falls Victim to a Software Supply Chain Vulnerability:
Safety Best Practices: OpenA Security Boulevar
Arbitrary Code Execution: Sny Plugin Review Process: OpenA
CS324 - Large Language Models: Stanford Universit Compromised PyTorch-nightly dependency chain: PyTorch
How data poisoning attacks corrupt machine learning models: CSO Onlin
ML Supply Chain Compromise: MITRE