TLDR: JSPLIT is a new framework designed to solve ‘prompt bloating’ in AI agents that use many external tools via the Model Context Protocol (MCP). By organizing tools into a hierarchical taxonomy and intelligently selecting only the most relevant ones based on a user’s query, JSPLIT significantly reduces prompt size, lowers computational costs, and improves tool selection accuracy, especially in environments with a large number of available tools. This allows AI agents to operate more efficiently and effectively.
The landscape of Artificial Intelligence is rapidly evolving beyond simple conversational models. Today, AI systems are transforming into sophisticated agents capable of interacting with a multitude of external tools and services. This shift, while empowering, introduces a significant challenge known as ‘prompt bloating’.
Prompt bloating occurs when the descriptions of numerous tools, necessary for an AI agent to function, are included directly within its prompt. As the number of available tools grows, these prompts become excessively long. This leads to several problems: increased computational costs, higher latency in responses, and a decrease in the agent’s ability to accurately select the most relevant tools for a given task.
To tackle this issue, researchers have introduced JSPLIT, a novel framework designed to optimize how AI agents manage their context when interacting with a large array of tools, particularly those following the Model Context Protocol (MCP). The MCP is a standardized interface that allows Large Language Models (LLMs) to connect with external data sources and functionalities, eliminating the need for custom integrations for every tool.
How JSPLIT Works
At its core, JSPLIT employs a taxonomy-driven approach. It organizes all available MCP tools into a hierarchical classification system, much like a library categorizes books by subject. Each category in this taxonomy has a clear, human-readable description of its functional scope.
When a user provides a query or task, the AI agent’s embedded LLM evaluates this query against the taxonomy’s class descriptions. Based on semantic relevance, JSPLIT intelligently selects only the taxonomy classes pertinent to the query. Subsequently, it filters the vast pool of MCP tools, including only those that belong to the identified relevant classes in the agent’s execution context. This selective process significantly reduces the prompt size, ensuring the agent receives only the information it needs for the task at hand, without compromising its ability to complete it effectively.
The Evolution of Taxonomies
JSPLIT’s development involved two main versions of its functional taxonomy: Taxonomy v1 and Taxonomy v2. Taxonomy v1 was an initial attempt, categorizing tools into eight broad functional areas with primary and optional secondary tags. Taxonomy v2, a more advanced version, expanded to eleven top-level categories, introducing deeper structure, clearer definitions, and mechanisms for handling tools that don’t fit neatly into existing slots.
Demonstrated Benefits
Extensive evaluations using a dataset of approximately 2,000 MCP servers and 200 user queries demonstrated JSPLIT’s effectiveness. The experiments simulated a ‘needle in a haystack’ scenario, embedding the correct tool among a varying number of irrelevant ‘noise’ tools.
The results were compelling: JSPLIT achieved a substantial reduction in input token costs, often by more than two orders of magnitude, compared to a baseline approach where all tool descriptions were injected into the LLM’s context. More importantly, as the number of available tools scaled into the hundreds, JSPLIT, particularly with Taxonomy v2, maintained a stable and significantly higher tool selection accuracy (around 69%) compared to the baseline, which saw its accuracy plummet below 40%.
Further studies also explored the impact of the LLM used for the internal classification process within JSPLIT. While larger API-based models offered marginal accuracy improvements, smaller API models provided an excellent balance of high accuracy and lower computational cost, suggesting practical deployment advantages.
Also Read:
- The Gatekeeper Protocol: A New Framework for Reliable and Efficient AI Agents
- KVCOMM: Boosting Multi-Agent LLM Efficiency with Smart Memory Reuse
Challenges and Future Directions
While highly effective, JSPLIT’s error analysis revealed some challenges, particularly in distinguishing between semantically similar tool categories (e.g., ‘Memory and Knowledge Management’ often confused with ‘Search and Information Retrieval’). This highlights the need for even more discriminative features and clearer category boundaries.
Looking ahead, the researchers plan to refine taxonomy descriptions, develop real-time classification mechanisms for new tools, and introduce an even more sophisticated Taxonomy v3, which aims for greater independence between categories and a separate dimension for domain classification. This ongoing work promises to further enhance the scalability and reliability of AI agents operating in complex, tool-rich environments.
For more technical details, you can read the full research paper here.
