TLDR: A new research paper reveals that Tool Invocation Prompts (TIPs), the instructions governing how LLM-based agentic systems interact with external tools, are a significant and overlooked security vulnerability. The study demonstrates that popular agentic systems are susceptible to TIP attacks, leading to Denial of Service (DoS) and Remote Code Execution (RCE) by manipulating tool descriptions and returns. Current defense mechanisms like guard models and self-reflection are often insufficient, highlighting the critical need for robust, layered security measures to protect these emerging AI systems.
Large Language Model (LLM)-based agentic systems are rapidly transforming various industries, from customer service to software engineering. These autonomous systems leverage powerful LLMs to understand complex user requests, make decisions, and execute actions by interacting with external tools. This ability to use tools is what allows them to perform multi-step reasoning tasks in real-world environments.
At the heart of this “LLM × Tools” interaction lies a crucial, yet often overlooked, component: the Tool Invocation Prompt (TIP). TIPs are essentially the instructions and communication protocols that define how an LLM interacts with external tools. They are designed by the agentic systems themselves to guide tool usage and even act as a security guard, ensuring the safety and correctness of information exchanged between the LLM and these tools.
Despite their critical role in security, TIPs have not been thoroughly investigated for potential vulnerabilities until now. A recent research paper, “Exploit Tool Invocation Prompt for Tool Behavior Hijacking in LLM-Based Agentic System”, sheds light on the significant security risks associated with TIPs. The study reveals that many popular LLM-based agentic systems are susceptible to TIP attacks, which can lead to severe consequences like Remote Code Execution (RCE) and Denial of Service (DoS).
Understanding the Threat: How TIPs Can Be Hijacked
The researchers developed a systematic workflow called TEW (TIP Exploitation Workflow) to demonstrate how external tool behavior can be hijacked by strategically manipulating tool invocation. The attacks primarily exploit two channels within the TIP structure: tool descriptions and tool returns.
One type of attack, a format-based untargeted attack, focuses on disrupting the system. By injecting malicious prompts into tool descriptions, the LLM can be tricked into generating malformed outputs that violate the system’s parsing requirements. This leads to a Denial of Service (DoS) condition, where the agent becomes unresponsive and unable to process tasks.
More severe are the logic-based targeted attacks, which aim for Remote Code Execution (RCE). These attacks come in two variants. RCE-1 directly embeds malicious code into tool descriptions, which can then be executed during the system’s initialization. RCE-2 is even more sophisticated, exploiting both tool descriptions and tool returns. Here, tool descriptions are modified to ensure the invocation of a malicious tool, and the actual harmful payload is injected into the tool’s return value. This multi-channel approach significantly expands the attack surface, allowing exploitation even in systems that resist direct injection.
Real-World Vulnerabilities Across Systems
The empirical evaluation covered a range of widely adopted LLM-based agentic systems, including Integrated Development Environments (IDEs) like Cursor and Cline, Command-Line Interfaces (CLIs) such as Claude Code, and chatbox agents like CherryStudio. The findings were stark: every tested client was vulnerable to at least one attack outcome.
DoS attacks were broadly observed across most agent categories. RCE-1 primarily affected IDE-based agents due to their deep tool integration and exposure to system-level execution channels. Notably, RCE-2 proved highly effective, succeeding in all RCE-1 vulnerable IDEs and even in Claude Code, which had resisted RCE-1. This highlights how exploiting tool-return pathways can bypass initial defenses.
The study also examined how different LLM backends (like GPT-5, Claude-sonnet-4, Gemini-2.5-pro, and Grok-4) influenced exploitability. While stronger alignment in newer models sometimes reduced attack success, client design choices and TIP integration still created varied outcomes. Less aligned backends often showed higher success rates for RCE attacks.
The Cost of Attack and Defense Challenges
The resource cost of these attacks, measured in token usage, varied. DoS attacks generally incurred moderate costs, while RCE-1 required larger payloads. RCE-2 was consistently the most resource-intensive due to its multi-channel reinforcement, often requiring thousands of injected tokens.
The research also explored potential defense mechanisms, including using guard models (smaller LLMs to filter prompts) and self-reflection strategies (where the LLM is instructed to check for malicious content). However, these preliminary defenses were found to be insufficient against more sophisticated TIP attacks. Guard models often failed to capture complex injection patterns, and self-reflection yielded inconsistent protection, especially against DoS attacks.
Also Read:
- Securing AI: Understanding and Defending Against Prompt Injection Attacks
- Invisible Threats: How Hidden Web Code Manipulates AI Summaries
Moving Forward: Strengthening Defenses
The findings underscore the urgent need for comprehensive security measures for TIPs. The researchers suggest a layered defense strategy that combines external filtering (like guard LLMs) with internal verification (such as reflection or anomaly detection). Adaptive filtering, continual learning, and consensus mechanisms (querying multiple defense models) are also recommended to build more robust systems. Furthermore, integrating provenance and trust signals could help flag prompts from untrusted sources for stricter scrutiny.
This groundbreaking research establishes a baseline for understanding and mitigating the systemic risks posed by Tool Invocation Prompts, paving the way for more secure and reliable LLM-based agentic systems in the future.
