TLDR: AGENT ARMOR is a novel framework designed to protect Large Language Model (LLM) agents from prompt injection attacks. It achieves this by converting the agent’s runtime traces into structured program dependency graphs. A property registry annotates these graphs with security metadata, and a type system then enforces fine-grained security policies, detecting and mitigating vulnerabilities by analyzing data flow and trust boundaries. Evaluated on the AgentDojo benchmark, AGENT ARMOR significantly reduces attack success rates while maintaining agent utility.
Large Language Model (LLM) agents are transforming how we solve problems by combining natural language understanding with the ability to use external tools. Imagine an AI assistant that can not only understand your complex requests but also browse the web, manage files, or interact with APIs to get things done. This flexibility, however, comes with significant security challenges, especially from prompt injection attacks.
Prompt injection is a type of attack where malicious instructions are subtly inserted into the agent’s input, tricking it into performing unintended or harmful actions. Because LLM agents often have broad access to tools and their internal reasoning isn’t always transparent, they can be vulnerable to these hidden commands, potentially leading to data leaks or unauthorized operations.
A new research paper introduces a novel solution called AGENT ARMOR, which tackles this problem from a unique perspective: by treating the agent’s runtime behavior as a structured program. This allows for the application of well-established program analysis techniques to enforce security policies.
AGENT ARMOR operates through three core components:
The Graph Constructor
This component takes the agent’s dynamic, language-driven interactions—like user prompts, the agent’s internal thoughts, tool selections, and tool outputs—and converts them into formal, graph-based representations. It builds a Control Flow Graph (CFG) to map the temporal and logical order of execution, a Data Flow Graph (DFG) to track how information propagates, and a unified Program Dependency Graph (PDG) that combines both. This transformation makes the agent’s otherwise opaque actions analyzable.
The Property Registry
While the graphs show relationships, they don’t inherently understand the security implications of tools or data. The Property Registry addresses this by annotating the graph nodes with rich metadata. It stores predefined information about known tools (like their parameters, side effects, and security labels) and data entities (such as their type, sensitivity, and trustworthiness). It also includes ‘scanners’ that can dynamically infer and register properties for new or unseen tools and data during runtime, ensuring the system remains robust in evolving environments.
Also Read:
- Securing LLMs: A Dual Approach to Combat Prompt Injection and Data Leaks
- Pro2Guard: Ensuring LLM Agent Safety Before Incidents Occur
The Type System
This is where security policies are enforced. The Type System assigns, propagates, and verifies security types across the Program Dependency Graph. For example, a user prompt might be labeled as ‘untrusted,’ while a system tool could be ‘high integrity.’ The system then ensures that sensitive data doesn’t flow to untrusted outputs or that low-integrity inputs don’t influence high-integrity actions. It performs checks both within individual actions and across the entire workflow, flagging or blocking behaviors that violate predefined security rules.
The researchers evaluated AGENT ARMOR on AgentDojo, a benchmark specifically designed for testing AI agent robustness against prompt injection attacks. The results were promising: AGENT ARMOR significantly reduced the attack success rate to just 1.16% on average, while maintaining the agent’s ability to complete legitimate tasks with only a minor reduction in utility (around 6.7%). It achieved a high true positive rate (TPR) of 95.75%, meaning it was very effective at detecting actual security violations.
While AGENT ARMOR represents a significant step forward, the paper also discusses some limitations. For instance, the accuracy of its dependency reasoning relies on LLMs, which aren’t always perfectly reliable. Tracking vast amounts of data, especially in complex scenarios like code or file system management, can also be challenging. Additionally, future work will focus on defending against highly adaptive attacks that might try to disturb the dependency reasoning process itself.
In conclusion, AGENT ARMOR offers a principled and verifiable approach to securing LLM agents by transforming their dynamic behaviors into structured programs that can be rigorously analyzed. This work opens a new direction for building more secure and trustworthy AI agent architectures. You can read the full research paper here: AGENT ARMOR: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection.
