TLDR: DATAMIND is a novel framework for creating generalist, open-source data-analytic AI agents. It addresses key challenges in data synthesis, training strategies, and stable code execution. By generating a high-quality dataset (DATAMIND-12K) and employing a dynamic SFT-RL training approach, DATAMIND-14B achieves state-of-the-art performance, surpassing leading proprietary and open-source models on various data analysis benchmarks. The research also provides insights into effective agent training.
Data analysis is a critical component of scientific discovery and innovation, and AI agents are increasingly being developed to automate this complex process. These agents are designed to process, model, and compute data by generating code, ultimately providing valuable insights for decision-making. However, a significant challenge has been the reliance on proprietary AI models and the struggle of open-source models to handle diverse data formats, large-scale files, and the multi-step reasoning required for real-world analytics.
A new research paper introduces DATAMIND, a comprehensive approach to building scalable and generalist open-source data-analytic agents. This work tackles three core challenges: the scarcity of high-quality data resources for training, the need for an effective training strategy, and the instability often encountered during code-based, multi-turn interactions.
The DATAMIND Approach
DATAMIND employs a sophisticated pipeline to overcome these hurdles. First, it addresses the data scarcity issue through a scalable data synthesis method. This involves collecting a wide variety of data files from sources like Kaggle, BIRD, and OmniSQL. To generate diverse and challenging questions, the system uses a fine-grained task taxonomy (categorizing data analysis into 18 types, such as correlation analysis, anomaly detection, and feature engineering) and a recursive easy-to-hard task composition mechanism. This means simple tasks are chained together to create more complex, multi-step analytical challenges.
Next, DATAMIND focuses on creating high-quality ‘expert trajectories’—step-by-step solutions to these synthesized questions. It uses a knowledge-augmented sampling strategy, where an expert AI model generates multiple solution paths. A ‘judge model’ then filters these, ensuring consistency in answers and selecting the most concise and accurate trajectory for training. This process also includes rule-based filtering to ensure format compliance, control answer length, and maintain linguistic integrity.
Advanced Training and Stable Execution
The training strategy for DATAMIND agents combines Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). Unlike traditional methods, DATAMIND uses a dynamically adjustable weighting factor between SFT and RL losses. This allows the model to first learn extensively from expert data (SFT) and then gradually shift to exploration and refinement through RL, ensuring stable and effective learning. The paper highlights that SFT acts as a crucial stabilizer for RL training, preventing performance degradation, but also notes that excessive SFT can hinder exploration.
To ensure stable code-based multi-turn interactions, DATAMIND implements several optimizations. It uses asynchronous interaction for model generation and code execution, reducing memory demands. A ‘chunked code maintenance’ method, similar to a notebook, allows the model to generate only necessary code snippets, saving memory. Furthermore, each trajectory runs in an isolated, sandboxed environment with strict limits on execution time and memory, enhancing security and stability.
Also Read:
- AI Agents Transform Data Analysis: A Comprehensive Overview
- InfoAgent: An Autonomous AI for Complex Information Seeking
Impressive Results and Key Insights
Using this pipeline, the researchers curated DATAMIND-12K, a high-quality training dataset of nearly 12,000 trajectories covering diverse tasks and data formats. Trained on this dataset, the DATAMIND-14B model achieved an average score of 71.16% on multiple data analysis benchmarks, outperforming strong proprietary baselines like DeepSeek-V3.1 and GPT-5. The smaller DATAMIND-7B model also performed exceptionally well among open-source alternatives, scoring 68.10%.
The study also yielded valuable insights for the AI community: self-consistency filtering is vital for trajectory quality, SFT loss can stabilize RL training but also cause rigidity if overused, and RL can narrow performance gaps between different base models but doesn’t fundamentally reverse their inherent capabilities. The researchers plan to release DATAMIND-12K and the DATAMIND-7B and 14B models to foster future research. You can find the full research paper here: SCALINGGENERALISTDATA-ANALYTICAGENTS.
