TLDR: A new research paper introduces MCPS ECBENCH, a benchmark that systematically identifies and tests 17 types of security attacks across four attack surfaces in the Model Context Protocol (MCP), which connects LLMs to external tools. Experiments on Claude, OpenAI, and Cursor reveal that over 85% of identified attacks successfully compromise at least one platform, with core vulnerabilities affecting all, highlighting urgent security needs.
Large Language Models (LLMs) are becoming central to many applications, connecting to external data and tools through something called the Model Context Protocol (MCP). While this connection makes LLMs much more powerful, it also opens up new doors for security risks and attacks.
A new research paper, “MCPS ECBENCH : A Systematic Security Benchmark and Playground for Testing Model Context Protocols,” by Yixuan Yang, Daoyuan Wu, and Yufan Chen, dives deep into these security challenges. The researchers have created the first comprehensive classification of MCP security, identifying 17 different types of attacks across four main areas where vulnerabilities can occur.
To systematically test these vulnerabilities, the team developed MCPS ECBENCH. This is a complete security benchmark and testing environment that includes datasets of prompts, special MCP servers, MCP clients, and scripts designed to carry out various attacks. It’s built to be flexible, allowing other researchers to add their own components for thorough security assessments.
The findings from their experiments are quite striking. Over 85% of the identified attacks successfully compromised at least one platform. Core vulnerabilities, such as flaws in the protocol itself or its implementation, were found to affect major platforms like Claude, OpenAI, and Cursor universally. This means fundamental weaknesses are present across the board.
However, the paper also notes that attacks based on user prompts or those targeting specific tools showed more varied success rates across different hosts and models. For instance, Claude was very good at blocking prompt injection attacks, while OpenAI and especially Cursor were more susceptible.
The research highlights several specific attack types. For example, “Tool/Service Misuse via ‘Confused AI'” showed that LLMs could be tricked into misusing tools if their purpose was misrepresented. “Data Exfiltration” attacks successfully leaked sensitive information about available tools across all tested platforms. “Sandbox Escape” attacks, which allow attackers to run commands on the host machine, also had a 100% success rate on all three MCP hosts.
Other notable attacks include “Package Name Squatting,” where malicious tools or servers with similar names to legitimate ones could confuse the LLM, and “Tool Poisoning,” where a malicious tool could be made to appear optimal for a task, leading the LLM to use it incorrectly.
The MCPS ECBENCH framework is designed to standardize how MCP security is evaluated, enabling rigorous testing across all layers of the protocol. The researchers have made their benchmark framework open and modular, encouraging future research in this critical area. You can find more details about their work and the benchmark in their paper available at this link.
Also Read:
- New Research Uncovers Extensive Vulnerabilities in AI Agent Communication Standard
- Unpacking How Large Language Models Interact with External Tools: A New Study’s Insights
This research underscores the urgent need for robust security measures in MCP-powered AI agent systems to prevent data breaches, unauthorized actions, and real-world harm as these systems become more integrated into sensitive environments.
