TLDR: A new research paper introduces the Chain of Action (CoA) framework, which unifies high-level planning and low-level control in AI agents for Minecraft. By treating abstracted actions as intermediate reasoning steps, CoA enables a single Vision-Language-Action (VLA) model to learn more robust and generalizable policies. The paper also presents OpenHA, an open-source suite including benchmarks, datasets, and models, demonstrating that training an agent on diverse action spaces leads to superior performance compared to specialized agents.
Developing artificial intelligence agents that can effectively operate in complex, open-ended environments like Minecraft has long been a significant challenge. A core hurdle lies in how these agents represent and execute actions. Recent research from Peking University and the National University of Singapore, in collaboration with Team CraftJarvis, introduces a novel framework called Chain of Action (CoA) and an accompanying open-source suite, OpenHA, to address this very issue.
Traditionally, AI agents have struggled with a critical dilemma: no single way of representing actions is universally best. Some actions are high-level, like ‘chop down a tree,’ while others are very low-level, such as individual keyboard presses or mouse movements. Existing approaches, whether modular systems or end-to-end Vision-Language-Action (VLA) models, often face limitations. Modular systems cannot learn and improve through continuous interaction, while VLA models find it hard to bridge the gap between abstract language commands and precise, low-level controls. Hierarchical agents, which separate high-level planning from low-level execution, also grapple with defining these abstract actions effectively.
Introducing the Chain of Action (CoA) Framework
The new paper presents the Chain of Action (CoA) framework as a solution that unifies high-level planning and low-level control within a single, monolithic VLA model. Instead of treating an abstract action as a command for a separate policy, CoA views it as an intermediate ‘reasoning step’ – much like a chain of thought – that guides the generation of the final, executable action. This means the agent first ‘thinks’ about a high-level plan, and then, conditioned on this thought, generates the precise low-level actions needed to carry it out. For example, to ‘chop a tree,’ the CoA agent might first generate a grounded action like ‘Approach(object=tree, coordinate=[X, Y, Z])’ and then follow with the necessary keyboard and mouse inputs to reach that position and perform the action.
This approach leverages the power of modern auto-regressive transformers, allowing the entire process to be trained end-to-end. It breaks down complex problems into more manageable, sequential steps, making the agent’s decision-making process more structured and easier to learn.
Diverse Action Spaces and All-in-One Training
The researchers conducted an extensive comparison of various action spaces, including primitive environmental actions (raw keyboard/mouse inputs), textual actions (descriptive strings), language skills (goal-oriented behaviors), motion primitives (object-agnostic movements), grounding actions (combining symbolic verbs with spatial coordinates), and latent actions (learned from data). Their findings confirmed that the optimal action space is highly task-dependent; what works best for navigation might not work for crafting.
To overcome this, the paper proposes an ‘All-in-One’ training strategy. By using the CoA framework, all high-level abstracted actions, regardless of their origin, are ultimately grounded in the same primitive environmental action space. This ‘common currency’ allows a single agent to be trained on a diverse mixture of action representations. The result is a highly versatile agent that not only understands and generates multiple types of actions but also develops a more robust and generalizable internal policy, consistently outperforming specialist agents trained on only one type of action.
Also Read:
- Advancing Robot Generalization Through Preserved Vision-Language Representations
- PDDL-INSTRUCT: Enhancing LLMs for Precise Symbolic Planning
Flexible Inference and Open-Source Resources
The CoA framework also offers flexibility at inference time, supporting two modes: a ‘Decoupled Inference Mode’ (Fast), which mimics traditional hierarchical agents by offloading low-level action decoding to a lightweight policy, and a ‘Unified Autoregressive Mode’ (Slow), where the single, unified model handles both planning and execution. While the slow mode is more computationally intensive, it consistently achieves higher success rates by applying the full reasoning capacity of the large model to both high-level planning and low-level execution.
To foster reproducible research and further advancements, the team has released the OpenHA suite. This comprehensive resource includes a benchmark of over 800 distinct tasks, curated datasets, source code, and all pre-trained model checkpoints. You can explore the OpenHA suite and its resources at their GitHub repository. The research paper provides further details on these advancements.
This work marks a significant step towards creating more capable and general-purpose autonomous agents. By synergizing multiple action representations within a unified framework, the OpenHA project demonstrates a promising path for AI agents to master diverse tasks across various domains, from virtual worlds like Minecraft to real-world applications.
