TLDR: HierSearch is a new AI framework for enterprise deep search that combines local and web information sources. It uses a hierarchical structure with specialized local and web search agents coordinated by a planner, trained with hierarchical reinforcement learning. A knowledge refiner filters out irrelevant or hallucinated information. This approach significantly improves search and reasoning performance and efficiency compared to traditional methods, especially in multi-source environments.
In the evolving landscape of artificial intelligence, the ability of large reasoning models (LRMs) to find and synthesize information is becoming increasingly crucial. This process, often termed “deep search,” goes beyond simple retrieval, leveraging the reasoning capabilities of these models to tackle complex information retrieval tasks. However, a significant challenge arises when these deep search systems need to access information from multiple sources – specifically, both private, local enterprise data and the vast, ever-changing public web.
Traditional deep search methods often fall short by being limited to a single knowledge source, whether it’s an internal database or the internet. Enterprises, however, require a more comprehensive solution that can seamlessly integrate both. A straightforward approach might be to train an AI agent with all available search tools using a technique called “flat reinforcement learning.” Yet, this method has proven inefficient, struggling with large action spaces and failing to effectively master complex tools, especially when dealing with difficult search scopes like the web.
Introducing HierSearch: A Hierarchical Approach to Deep Search
To overcome these limitations, researchers have proposed a novel framework called HierSearch. This innovative system employs a hierarchical agentic deep search paradigm, trained using hierarchical reinforcement learning (HRL). Imagine a team of specialized agents working together, coordinated by a central planner.
At the lower level, HierSearch features two specialized agents:
- Local Deep Search Agent: This agent is designed to interact with internal, private knowledge sources. It can access a local text chunk corpus (collections of text segments) and a local knowledge graph (a structured network of facts). Its tools include “chunk search” for relevant text passages and “graph search” to retrieve factual triplets, along with “get adjacent passages” to find related text from entities in the knowledge graph.
- Web Deep Search Agent: This agent is responsible for navigating the public internet. It utilizes “web search” to query search engine APIs for links, titles, and snippets, and “browse url” to access and extract relevant information from actual web pages.
Overseeing these specialized agents is the high-level Planner Agent. This central intelligence drafts search plans, coordinates the low-level agents, analyzes the evidence they provide, and ultimately formulates the final answer. The low-level agents are essentially “packaged” as tools for the planner, allowing it to call them individually or even both simultaneously via an “all search agent” tool.
Smart Training and Knowledge Refinement
The training of HierSearch is a two-stage process using Hierarchical Reinforcement Learning. First, the low-level local and web deep search agents are trained independently. This allows them to master their specific domain tools efficiently, as the number of tools is limited and closely related. Once these agents are proficient, the high-level planner agent is trained. Because the deep search agents are already well-trained, they abstract away the complex interactions with individual search tools, making it much easier for the planner to learn search planning and knowledge integration across diverse sources.
A crucial component of HierSearch is its Reasoning-Aware Knowledge Refiner. When low-level agents return information, they might include irrelevant search results or even “hallucinations” (incorrectly generated information). Directly feeding all this raw data to the planner could lead to errors or inefficient processing. The knowledge refiner addresses this by filtering out key evidence that directly contributes to the agent’s thinking process and overall conclusion. It does this in two steps: first, by selecting evidence relevant to the immediate next thought process, and second, by identifying evidence that contributes globally to the agent’s conclusion, even if not immediately used.
Also Read:
- Understanding Advanced AI Search Agents
- Enhancing AI Agent Tool Selection with Knowledge Graphs for Enterprise Tasks
Performance and Efficiency
Extensive experiments across various benchmarks, including general, financial, and medical domains, have shown that HierSearch significantly outperforms existing deep search and multi-source retrieval-augmented generation baselines, as well as the flat reinforcement learning approach. It achieves better performance without a substantial increase in search tool calls, particularly expensive web searches.
For instance, HierSearch demonstrates superior search and reasoning success rates compared to other specialized deep search agents. While local search is generally easier, HierSearch effectively leverages web search to supplement knowledge, outperforming flat RL in web search success rates, which often struggles with the wider scope and noise of web data. The hierarchical design also proves computationally efficient, avoiding the unnecessary and costly web search tool calls often seen in parallel search methods.
In conclusion, HierSearch represents a significant advancement in enterprise deep search by effectively integrating local and web knowledge sources through a hierarchical agentic framework. Its innovative design, coupled with hierarchical reinforcement learning and a reasoning-aware knowledge refiner, enables more effective and efficient information retrieval and reasoning across diverse domains. For more in-depth details, you can refer to the full research paper here.
