TLDR: CODA is a new trainable framework for autonomous GUI agents that combines a generalist planner (Cerebrum) with a specialist executor (Cerebellum), inspired by the human brain. It uses a two-stage training process: first, individual specialization via decoupled reinforcement learning (GRPO) for specific applications, and then generalization through supervised fine-tuning on aggregated successful trajectories. This approach allows the agent to excel in both long-horizon planning and precise execution in complex scientific software, outperforming existing models by learning from experience without extensive human-labeled data.
In the rapidly evolving world of artificial intelligence, autonomous agents designed to interact with Graphical User Interfaces (GUIs) are becoming increasingly important. These agents aim to automate a wide range of digital tasks, from scientific computing to engineering analysis. However, developing agents that can handle both complex, long-term planning and precise, fine-grained execution in specialized software environments has been a significant challenge.
Traditional approaches often fall into one of two categories: generalist agents that are good at planning but struggle with precise actions, or specialized agents that excel at execution but lack robust high-level planning capabilities. While some frameworks try to combine a “planner” and an “actor,” they often remain static and cannot learn from experience, which is a major limitation, especially in scientific fields where high-quality data is scarce.
To overcome these hurdles, researchers have introduced a new framework called CODA. This innovative system is trainable and draws inspiration from the human brain’s structure, specifically the division of labor between the cerebrum and the cerebellum. CODA synergizes a generalist planner, referred to as the Cerebrum, with a specialist executor, called the Cerebellum. This dual-brain approach allows for both strategic thinking and precise action execution.
The CODA framework employs a unique two-stage training pipeline. The first stage, known as Specialization, focuses on training an expert planner for each scientific application individually. This is achieved using a decoupled Group Relative Policy Optimization (GRPO) approach, which effectively bootstraps learning from a small initial set of task trajectories. This stage ensures that the planner becomes highly proficient in specific domains.
Following specialization, the second stage, Generalization, comes into play. Here, all successful trajectories generated by the specialized experts are aggregated into a consolidated, high-quality dataset. This rich dataset is then used to perform supervised fine-tuning (SFT) on the final planner. This process equips the planner with robust, cross-domain capabilities, transforming it into a true generalist that can adapt to various software environments.
The architecture of CODA integrates existing powerful models: Qwen2.5-VL serves as the Planner (Cerebrum), responsible for generating high-level thoughts and plans based on interaction history and screenshots. UI-Tars-1.5 acts as the Executor (Cerebellum), translating these abstract plans into precise, executable GUI actions, such as clicking specific coordinates or typing text. A key advantage of this decoupled approach is its data efficiency; only the planner is optimized for domain adaptation, while the executor, already possessing strong generalization abilities from massive pretraining, remains fixed.
Training the planner in this cerebrum-cerebellum separation avoids the need for costly human-labeled trajectories. Instead, CODA leverages an automated judging system built from open-source models to provide dense reward signals. This, combined with autonomous interaction with scientific software environments via the static executor, allows the planner to gradually acquire domain-specific planning abilities with zero human effort. Furthermore, a distributed virtual machine system accelerates the interaction process across multiple software environments in parallel, making reinforcement learning more efficient.
Evaluations on four challenging applications from the ScienceBoard benchmark demonstrate that CODA significantly outperforms baseline models and sets a new state-of-the-art among open-source models. This success confirms the effectiveness of combining stable execution with adaptive planning. For those interested in exploring the framework further, the models and code are openly available. You can find more details in the full research paper: CODA: Coordinating the Cerebrum and Cerebellum for a Dual-Brain Computer Use Agent with Decoupled Reinforcement Learning.
Also Read:
- MUA-RL: Training Language Agents for Dynamic User Conversations
- HIPLAN: Enhancing LLM Agents with Adaptive Hierarchical Planning for Complex Tasks
The Importance of a Robust Judging System
A crucial component of CODA’s reinforcement learning framework is its accurate judging system, which provides reliable reward signals for agent trajectories. The researchers conducted detailed evaluations of this judge model, showing improved precision in decision-making. Several strategies were employed to enhance the judge’s accuracy, including voting-based methods, using multi-resolution inputs to capture both global and fine-grained execution dynamics, and model ensembling. These advancements ensure that the planner receives high-quality feedback, which is essential for improving its reasoning and learning domain-specific knowledge.
