10 most critical LLM vulnerabilities

Enterprise adoption of generative AI technologies has exploded due to the rapid evolution of the technology and the emergence of a variety of business use cases. But large language models (LLMs) can accidentally produce harmful results, leak information, or become exposed to threat actors. These vulnerabilities are changing as the technology evolves and as attackers find new ways to compromise systems. For enterprises, this means the risk of bad publicity, compliance or cybersecurity exposure, legal liability, or even class-action lawsuits. To keep up with the latest in LLM risks, the Open Worldwide Application Security Project (OWASP) has updated its list of the top 10 most critical vulnerabilities of LLM applications. For the first time, the list is influenced by real-world incident data alongside expert voting on classes of AI security risk. Prompt injection and sensitive information disclosure remain the two most severe threats, with excessive agency (agentic actions outside permitted bounds) rising to OWASP’s No. 3 spot, a change reflecting the evolution from chatbots to agentic systems that call APIs and run code. Issues such as unbounded consumption of AI resources have grown in prominence while risks from improper output handling have dropped to the bottom of the list thanks to more widespread use of output sanitization. The changes reflect how fast the generative AI sector is evolving — and the list is likely to change again soon as new technologies hit the market. The list brings together the judgment of the security practitioners who attack and defend these systems along with the record of what has actually gone wrong in the field, each cross-referenced against each other, according to OWASP. The list intends not only to educate developers and organizations about potential security risks but also to suggest remediation strategies for improving the security posture of LLM applications. Organizations should work on all 10 categories, starting at the top. 1. Prompt injection Prompt injection has topped OWASP’s list since it was first released in early 2023. The attack technique involves maliciously crafted inputs meant to manipulate an LLM to unknowingly execute an attacker’s intentions. This can include direct “jailbreaking” or the manipulation of external inputs such as files or websites, potentially resulting in data exfiltration, social engineering, or other issues. For example, a user can write a prompt to trick a company chatbot into revealing proprietary information the user doesn’t have access to or upload a text, image, or audio file with hidden instructions to cause the application to misbehave. Prompt injection also includes cross-modal attacks aimed at manipulating tool output, intermediate reasoning, or persistent memory to nefarious ends. To reduce the risk of prompt injection, OWASP recommends the following: Constrain the model’s role and capabilities in the system prompt. Add a human in the loop for the most sensitive operations, requiring an extra approval step to check for unauthorized actions. Define a strict output schema, validating every response with trusted application code. Scan inputs and outputs (text, image, audio, and structured data) for harmful content and block sensitive or unauthorized content before it reaches the model or is returned to users. Hold credentials and state-change capability in application code, not the model, and grant least privilege per operation. 2. Sensitive information disclosure Also referred to as “data leakage,” sensitive information disclosure involves an LLM revealing protected data, proprietary algorithms, or other confidential details on output, which can result in unauthorized access, privacy violations, and other security breaches. Sensitive data can get into an LLM through multiple routes, including initial training, during fine-tuning or embedding, or when entered by a user’s prompt. Once the model has access to this information, there’s the potential for other, unauthorized users to see it. Preventative measures include: Use data sanitization and scrubbing to prevent LLMs from getting access to sensitive data during training or inference. Apply filters to user inputs to prevent sensitive data from being uploaded or to identify and remove or mask confidential information. Use strict access controls and the principle of least privilege when LLMs need to access data sources during inference. 3. Excessive agency This vulnerability has moved up from sixth place as agentic systems have become more commonplace in the enterprise. Excessive agency involves an LLM with too much power to perform actions and typically stems from excessive functionality, excessive permissions, and insufficient oversight. Damaging actions could be performed when an LLM hallucinates, when it falls victim to a prompt injection, a malicious plugin, poorly written prompts, or just because it’s a badly performing model, OWASP says. Excessive agency’s jump on OWASP’s lis