Jon Green
Software Engineer Intern
Bahar Shah
Staff Engineer
AI coding tools help developers build and deploy LLM applications quickly, but this speed comes with new security risks. Traditional static application security testing (SAST) tools that are pattern based weren’t designed to detect LLM-specific issues such as prompt injection sinks and exposed system prompts. These vulnerabilities often don’t become apparent until applications are already running in production, when remediation is more difficult and expensive.
Datadog Code Security’s AI-native SAST offers coverage for the OWASP Top 10 for LLM Applications to help you detect LLM-specific risks earlier in the development cycle. Developers get actionable feedback directly in the same workflows where they review and ship code, helping them remediate vulnerabilities faster.
In this post, we’ll explain:
What is the OWASP Top 10 for LLM Applications?
The OWASP Top 10 for LLM Applications is an industry-standard reference that describes vulnerability classes specific to applications that use LLMs. The list is the LLM equivalent of the traditional OWASP Top 10. It covers risks related to how user input reaches a model, how model outputs are handled downstream, whether LLM-driven actions are authorized, and more.
The following table summarizes Datadog Code Security’s AI-native SAST coverage within each risk in the OWASP Top 10 for LLM Applications:
| Risk | Examples of vulnerabilities detected by AI-native SAST |
|---|---|
| LLM01:2026 Prompt Injection | Injecting malicious input into an LLM prompt to manipulate the model into treating attacker-controlled data as instructions |
| LLM02:2026 Sensitive Information Disclosure | Exposing private data such as personally identifiable information (PII) or credentials through model outputs, training data, or API responses |
| LLM03:2026 Excessive Agency | Granting an LLM the ability to take consequential actions, such as deleting files or calling external APIs, without verifying that the user authorized those actions |
| LLM04:2026 Supply Chain | Introducing security risks through untrusted third-party models, plugins, or compromised training datasets |
| LLM05:2026 Data and Model Poisoning | Loading untrusted model files through unsanitized user input, enabling arbitrary code execution through malicious model files |
| LLM06:2026 Unbounded Consumption | Failing to limit model resource usage, allowing attackers to trigger excessive inference costs or degrade service availability |
| LLM07:2026 Misinformation | Returning LLM output without disclaimers or attribution, leaving downstream systems or users unable to verify the accuracy of model-generated content |
| LLM08:2026 Hidden Context Exposure | Exposing hidden context such as system prompts, tool schemas, RAG-retrieved content, and developer instructions through logs, error messages, or API responses, revealing information not intended for users |
| LLM09:2026 Vector and Embedding Weaknesses | Passing unsanitized user input directly into embedding queries or retrieval calls, allowing attackers to manipulate what content the model retrieves |
| LLM10:2026 Improper Output Handling | Using unvalidated model output directly in downstream systems, enabling injection attacks such as cross-site scripting and command injection |
How Datadog’s AI-native SAST detection works
Traditional SAST rules require a trade-off. If you make the rules too broad, they produce noise. If you make them too narrow, they miss real vulnerabilities.
Datadog Code Security’s AI-native SAST goes beyond rule matching by using LLMs to reason about code context and data flow. It then independently verifies candidate findings. Findings surface directly in the Datadog platform as pull request (PR) comments and through CI checks via PR Gates.
AI-native SAST coverage for the OWASP Top 10 for LLM Applications is available across Python, Go, Java, C#, TypeScript, and JavaScript. The following examples show vulnerable application code and illustrate the detection approaches for three of these risks:
Taint analysis detection for prompt injection
Prompt injection (LLM01) occurs when external input is injected directly into an LLM prompt without sanitization, allowing the model to treat attacker-controlled data as an instruction rather than content. An attacker can use this method to bypass safety guardrails, exfiltrate data, or manipulate the model into taking actions that the model was never intended to take.
AI-native SAST uses taint analysis to identify paths where untrusted input reaches an LLM call. Consider a Go handler that reads a query parameter and concatenates it directly into a prompt:
In this example, userQuery
is an untrusted source, and the LLM call is the sink. AI-native SAST can trace the flow between them and flag any path where the value reaches the prompt without appropriate handling or validation. This analysis focuses the finding on the data flow that creates the risk instead of matching only the presence of an LLM call or user input.
Control flow analysis detection for excessive agency
Excessive agency (LLM03) occurs when an LLM takes consequential actions without having the appropriate authorization checks in place. Such actions include executing code, calling external APIs, and modifying data.
Detecting excessive agency requires control flow analysis because AI-native SAST needs to determine what capabilities the LLM is granted before it can act. Consider this Python example, where an LLM-powered agent is initialized with tools that grant unrestricted access to the shell and file system:
In this example, create_agent
grants the agent unrestricted shell and file system access regardless of who the user is or what they are authorized to do. AI-native SAST can identify that no capability restrictions or authorization checks are applied before the agent is returned with these tools.
In a larger application, the same agent configuration might be reachable through multiple callers or conditional branches. Control flow analysis enables AI-native SAST to examine those paths and flag cases where at least one path initializes the agent without the required restrictions in place.
Pattern matching detection for hidden context exposure
Hidden context exposure (LLM08) is the revealing of sensitive information that is not intended for users. This information can include system prompts, tool schemas, RAG-retrieved content, and developer instructions that are written to logs or returned in API responses.
AI-native SAST uses pattern matching to identify code that exposes system prompt contents. For example, the following Java method writes its system prompt to an application log:
AI coding assistants can introduce this type of exposure by generating debug logging that includes system prompt contents. In this example, systemPrompt
does not originate from untrusted input. The value is sensitive because it contains instructions that configure the LLM application’s behavior. AI-native SAST can identify that systemPrompt
is passed directly to logger.info()
, where its contents could be exposed.
Start detecting LLM vulnerabilities with AI-native SAST
AI-native SAST in Datadog Code Security helps developers identify LLM-specific vulnerabilities before they reach production. Techniques such as taint analysis, control flow analysis, and pattern matching identify risks that require context beyond conventional static rules. To learn more, read the AI-native SAST documentation.
If you don’t have a Datadog account, you can sign up for a 14-day free trial to get started detecting vulnerabilities in your LLM applications.