A Unified Data Approach for Training Advanced AI Agents

TLDR: The Agent Data Protocol (ADP) is a new standardized representation language that unifies diverse datasets for training Large Language Model (LLM) agents. It addresses the fragmentation of existing agent training data by providing a simple, expressive schema for actions and observations. ADP streamlines the data conversion process, significantly reducing the effort required to integrate new datasets and agent frameworks. Experiments show that fine-tuning LLM agents with ADP-standardized data leads to substantial performance improvements (around 20% average gain) across various tasks like coding, web browsing, and tool use, achieving state-of-the-art results and demonstrating strong cross-task generalization.

The world of AI agents is rapidly expanding, with models capable of performing complex sequential tasks like coding, browsing, and using various tools. However, a significant hurdle has been the lack of standardized training data. Researchers often face a fragmented landscape where agent training datasets exist in numerous, inconsistent formats, making it incredibly difficult to combine them, share insights, or leverage them effectively for large-scale fine-tuning of Large Language Models (LLMs).

A new research paper introduces a groundbreaking solution to this problem: the Agent Data Protocol (ADP). ADP is a lightweight, unified representation language designed to act as an “interlingua” – a common language – that bridges the gap between diverse agent datasets and streamlined agent training pipelines.

What is the Agent Data Protocol (ADP)?

The core idea behind ADP is to simplify and standardize how agent interactions are recorded and structured. Instead of each dataset having its unique format for actions and observations, ADP provides a universal schema. This approach directly tackles the challenges of data complexity, heterogeneity, and the difficulty of comparing different datasets.

ADP is built on three key design principles:

• Simplicity: It offers a straightforward framework that eliminates the need for specialized engineering for each dataset, making large-scale agent data utilization accessible.
• Standardization: It unifies existing agent training datasets into a consistent format.
• Expressiveness: It’s designed to capture complex agent trajectories accurately without losing critical information, covering a wide variety of tasks from API usage to web browsing.

How ADP Works: Actions and Observations

At its heart, ADP represents every agent trajectory as a sequence of actions taken by the agent and observations received from the environment. These are categorized into distinct types:

• Actions:
  • API Actions: These are function calls with structured parameters, capturing tool use. For example, navigating to a website would be an APIAction like goto(url=https://www.google.com).
  • Code Actions: These represent code generation and execution in various programming languages, such as print("Hello World") in Python.
  • Message Actions: These are natural language communications between the agent and users, like an agent asking, “How can I help you?”
• Observations:
  • Text Observations: These capture text-based information from sources like user instructions or environmental feedback.
  • Web Observations: These represent the state of webpages, including raw HTML content, accessibility trees, URLs, and even screenshots, enabling support for complex browsing scenarios.

A Streamlined Conversion Pipeline

The researchers implemented a three-stage conversion pipeline to transform raw, heterogeneous datasets into training-ready formats using ADP:

1. Raw to Standardized: Original datasets are converted into the ADP schema, mapping their specific actions and observations to ADP’s standardized space.
2. Standardized to SFT (Supervised Fine-Tuning): ADP-standardized trajectories are then converted into a format suitable for specific agent frameworks (e.g., OpenHands, SWE-Agent, AgentLab), adapting to each framework’s unique architecture and interaction styles.
3. Quality Assurance: Automated validation ensures data correctness and consistency, verifying tool call formats and conversation structures.

This pipeline significantly reduces the effort required to integrate new datasets or agent frameworks. Without ADP, integrating D datasets with A agent frameworks would require D x A custom converters (quadratic effort). With ADP, it becomes a linear effort (D + A), as each dataset only needs one conversion to ADP, and each agent framework only needs one conversion from ADP.

Impressive Experimental Results

To demonstrate ADP’s effectiveness, the researchers unified 13 existing agent training datasets, creating the largest publicly available dataset for agent training, comprising 1.3 million trajectories. They fine-tuned various LLMs (Qwen2.5-7B-Instruct, Qwen3-8B, etc.) using ADP-standardized data across different agent frameworks and evaluated them on benchmarks like SWE-Bench (for coding), WebArena (for browsing), AgentBench OS (for operating system tasks), and GAIA (for general AI assistance).

The results were remarkable: ADP fine-tuning consistently led to substantial performance gains, averaging around 20% over corresponding base models. In many cases, ADP-trained agents achieved state-of-the-art or near-state-of-the-art performance without any domain-specific tuning. The study also highlighted significant benefits from cross-task transfer, where training on the diverse ADP data improved performance more than training on individual, task-specific datasets.

Also Read:

• MCP-Flow: Automating LLM Agent Tool Use in Dynamic Environments
• FunReason-MT: Enhancing AI’s Ability to Use Tools in Complex Conversations

Looking Ahead

The Agent Data Protocol represents a significant step towards making large-scale, reproducible, and scalable agent training more accessible to the research community. By providing a common language for agent data, ADP promises to unlock new avenues for progress in AI agent development.

For more in-depth information, you can read the full research paper: AGENTDATAPROTOCOL: UNIFYINGDATASETS FOR DIVERSE, EFFECTIVEFINE-TUNING OFLLM AGENTS.