TLDR: ReAcTree is a novel hierarchical task-planning framework for large language models (LLMs) that tackles complex, long-horizon tasks by dynamically constructing an agent tree. It uses LLM-powered agent nodes for reasoning, acting, and subgoal expansion, coordinated by control flow nodes (sequence, fallback, parallel). Enhanced by episodic memory for subgoal-level examples and working memory for shared observations, ReAcTree significantly improves task success rates and reduces errors compared to monolithic planning approaches, even enabling smaller LLMs to achieve strong performance in partially observable environments.
Recent breakthroughs in large language models, or LLMs, have opened up exciting new possibilities for how autonomous agents can make decisions and plan tasks. However, these powerful AI systems often struggle with very complex, long-term tasks. The main issue is that they typically try to solve the entire task in one go, using a single, continuous stream of past decisions and observations. This monolithic approach can lead to errors and makes it hard for LLMs to manage intricate challenges.
Addressing this limitation, a new framework called ReAcTree has been introduced. ReAcTree offers a hierarchical approach to task planning, designed to break down a complex overall goal into smaller, more manageable subgoals. It does this by dynamically building an ‘agent tree’ where each part of the tree plays a specific role in solving the task.
How ReAcTree Works
At the heart of ReAcTree are two main types of nodes: agent nodes and control flow nodes. Each agent node is powered by an LLM and is responsible for a specific subgoal. These nodes can perform three key functions: reason about the task, take actions in the environment, or expand the tree further by proposing new subgoals if the current task is too complex. This expansion is crucial for tackling long-horizon tasks effectively.
Inspired by concepts from Behavior Trees, control flow nodes act as coordinators. They dictate how the child agent nodes are executed and how their outcomes influence the overall plan. ReAcTree supports three types of control flow:
- Sequence: Executes subgoals one after another. If any subgoal fails, the entire sequence stops.
- Fallback: Tries subgoals in order until one succeeds. If a subgoal is successful, the remaining ones are skipped.
- Parallel: Executes multiple subgoals simultaneously, allowing independent progress.
This hierarchical structure is a significant advantage. By isolating subgoals, each agent node can focus on its specific context and goal, which dramatically reduces the chances of errors and hallucinations that can occur in long, undifferentiated planning trajectories. It also makes complex tasks more manageable for LLMs.
Enhanced with Memory Systems
To further boost its capabilities, ReAcTree integrates two complementary memory systems:
- Episodic Memory: This system stores past experiences at the subgoal level. When an agent node needs to make a decision, it retrieves relevant, goal-specific examples from this memory. This targeted retrieval helps the LLM learn more effectively from past successes and improves decision accuracy.
- Working Memory: Acting as a shared blackboard, working memory allows all agent nodes to exchange critical, environment-specific observations. For instance, it can track the latest locations of movable objects, reducing redundant searches and mitigating hallucinations by providing accurate, up-to-date situational awareness.
Also Read:
- Language Models Powering Smarter Multi-Agent Collaboration
- Optimizing LLM Collaboration: A Graph-Based Approach to Test-Time Scaling
Impressive Performance
Experiments conducted on challenging household task environments, WAH-NL and ALFRED, under partially observable settings, have shown that ReAcTree consistently outperforms strong task-planning baselines like ReAct across various LLMs. For example, on the WAH-NL dataset, ReAcTree achieved a 61% goal success rate with Qwen 2.5 72B, nearly doubling ReAct’s 31% with the same model size. Even with much smaller models, ReAcTree demonstrated strong performance, highlighting its efficiency and ability to make smaller LLMs more capable.
The framework’s ability to decompose tasks into simpler subgoals and retrieve focused examples allows even less powerful models to reason more effectively, significantly narrowing the performance gap between different model scales. This robust performance extends to unseen environments and tasks requiring precise, multi-step procedures, such as cooking a potato slice before placing it in a fridge.
While ReAcTree marks a significant step forward, the researchers acknowledge ongoing challenges such as LLM hallucinations, recognizing failures, and refining subgoal correction. Future work aims to address these areas to further enhance the framework’s robustness and applicability in real-world scenarios. You can read the full research paper here: ReAcTree: Hierarchical LLM Agent Trees with Control Flow for Long-Horizon Task Planning.
