TLDR: Researchers from Renmin University of China and Xiaohongshu have unveiled DeepAgent, a groundbreaking AI agent that integrates autonomous thinking, dynamic tool discovery, and action execution into a single, coherent reasoning process. This innovative approach, featuring an ‘Autonomous Memory Folding’ mechanism and a novel reinforcement learning strategy called ‘ToolPO’, enables DeepAgent to tackle complex, long-horizon tasks with scalable toolsets, marking a significant advancement in autonomous AI.
A new era for artificial intelligence agents has dawned with the introduction of DeepAgent, a sophisticated deep reasoning AI agent developed by a collaborative team from Renmin University of China and Xiaohongshu. This innovative framework redefines how AI agents interact with complex environments by performing autonomous thinking, dynamic tool discovery, and action execution within a unified, end-to-end reasoning process.
Traditional AI agent frameworks often rely on predefined ‘Reason, Act, Observe’ loops, which can become inefficient and limited when dealing with extensive toolsets or prolonged, multi-step tasks. DeepAgent, however, breaks away from this rigid paradigm, allowing the agent to maintain a global perspective on the entire task and dynamically adapt its strategy. The core innovation lies in its ability to output four distinct action types directly in text: internal thought, tool search, tool call, and memory fold.
One of DeepAgent’s most critical features is its ‘on-demand tool discovery’ mechanism. Instead of being pre-loaded with a fixed set of tools, DeepAgent queries a dense index containing descriptions from vast registries, such as over 16,000 RapidAPI tools and 3,912 ToolHop tools. This dynamic access ensures the agent can discover and utilize the most relevant tools as needed, aligning with real-world scenarios where available tools can change or expand.
To address the challenge of long-horizon tasks and the potential for context length explosion, DeepAgent incorporates an ‘Autonomous Memory Folding’ mechanism. When the model determines it’s necessary, an auxiliary Large Language Model (LLM) compresses the agent’s full interaction history into three structured memories: Episodic Memory (recording task events), Working Memory (tracking current sub-goals and recent issues), and Tool Memory (logging tool names, arguments, and outcomes). This intelligent compression allows the agent to continue from a compact yet information-rich state, preventing error accumulation and preserving critical context over extended interactions.
Furthermore, the researchers developed a novel reinforcement learning strategy called ‘Tool Policy Optimization’ (ToolPO) to teach robust and general-purpose tool use. Unlike supervised traces, which often struggle with the sparse nature of correct tool calls within long generations, ToolPO runs rollouts on LLM-simulated APIs. This approach ensures stable training and applies fine-grained credit attribution to tool invocation tokens, rewarding both final task success and accurate intermediate tool calls.
Also Read:
- Adaptive AI for Business: Introducing Dingtalk-DeepResearch, a Unified Multi-Agent Framework
- InfoFlow: Enhancing AI Agent Search Through Optimized Reward Feedback
Extensive experiments have demonstrated DeepAgent’s superior performance across a wide array of benchmarks. It consistently outperforms baseline models in general tool-use tasks, including ToolBench, API-Bank, TMDB, Spotify, and ToolHop, as well as in complex downstream applications like ALFWorld, WebShop, GAIA, and HLE. This robust performance underscores DeepAgent’s potential to significantly advance the capabilities of autonomous AI agents for real-world applications.
