TLDR: GraSP is a new framework that uses a graph-based, low-code approach to generate high-quality synthetic data for training large language models (LLMs) like those used for Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). It addresses challenges in data generation by offering scalable, modular, and fault-tolerant pipelines with features like multimodal input support, reusable workflow components, agentic execution, and automated quality tagging, significantly speeding up data preparation.
The rapid evolution of large language models (LLMs) hinges significantly on the availability of vast amounts of high-quality data. This data is crucial for various training paradigms, including Supervised Fine-Tuning (SFT) and alignment tasks like Direct Preference Optimization (DPO). However, obtaining such data from real-world sources often presents challenges related to cost, potential biases, and limited availability. This is particularly true for specialized applications like instruction tuning, tool-use supervision, and multi-agent interactions, where precise control over data structure and diversity is essential.
Synthetic data, generated through LLMs and automated pipelines, offers a promising alternative, providing greater flexibility and control. Yet, generating this data at scale introduces its own set of complexities. These include designing intricate, branching workflows, orchestrating diverse model backends and APIs, ensuring data validation and schema compliance, and enabling fault-tolerant execution through features like resumability and sharding.
Introducing GraSP: A Unified Framework
To tackle these challenges, researchers from ServiceNow Inc. have introduced GraSP, a general-purpose framework designed for scalable synthetic data generation. GraSP stands for a unified graph-based framework that simplifies the generation, quality tagging, and management of synthetic data for LLM training. It combines a low-code, YAML-based configuration approach with modular, graph-driven orchestration, allowing for the creation of complex workflows that include branching, looping, and conditional logic.
The framework is built on three core principles: Scalability, Modularity, and Reusability. Scalability is achieved through streaming data sources, resumable jobs, and support for various output formats. Modularity is ensured by YAML-defined Directed Acyclic Graph (DAG) workflows with conditional logic. Reusability is fostered through versioned, reusable graphs, nodes, and validators.
Key Innovations of GraSP
GraSP brings several key features to the forefront of synthetic data generation:
-
Low-Code, Modular Graph Configuration: It offers a user-friendly YAML-based interface, combined with LangGraph-style agents and a custom DAG engine, to define complex workflows concisely and extensibly.
-
Reusable Recipes (Subgraphs): A significant advantage is the ability to reuse common graph components across different tasks. This promotes modularity and reduces redundancy, allowing developers to build upon established patterns.
-
Multimodal Support: GraSP goes beyond text, natively handling audio and image inputs alongside text. It transparently loads various media formats, encodes them for LLM API compatibility, and supports multiple media fields per record, opening doors for tasks like speech recognition and visual QA.
-
Agentic Execution: The framework enables the creation of autonomous, tool-using agents based on the ReAct paradigm. These agent nodes extend LLM capabilities with dynamic tool invocation, multi-turn reasoning, and conditional decision-making, facilitating exploratory and interactive tasks.
-
Structured Output Generation: GraSP provides a flexible system for generating and validating structured outputs from LLMs. It supports both class-based and YAML-defined schemas, ensuring reliable formats and reducing post-processing efforts.
-
Resumability: For long-running jobs, GraSP offers fault-tolerant execution. In case of a failure, execution can resume from the last recorded checkpoint, preserving partial progress and saving computational resources.
-
Filterable OASST-Compatible Formatting and Quality Tagging: Outputs can be structured in an OASST-compatible format, making them easy to filter, inspect, and integrate into training pipelines. Additionally, a dual-stage quality tagging mechanism, combining heuristic rules and LLM-based evaluations, automatically filters and scores data to ensure high quality.
Also Read:
- Boosting Search System Training with Smarter Synthetic Data Generation
- DeepThink3D: A Framework for Enhanced 3D Situated Reasoning with LLMs
Impact and Future Outlook
The impact of GraSP is evident in its performance. Benchmarking against naive, single-threaded pipelines, GraSP demonstrated a 3x to 4x reduction in wall-clock time for representative code-generation tasks. This efficiency gain allows for rapid, large-scale data generation, significantly accelerating the development of custom LLMs for enterprise and research applications.
While GraSP currently focuses on text-only outputs (with multimodal output support planned), and has basic agent support, future work aims to enhance agent coordination, introduce stateful nodes, enable real-time graph editing, and incorporate uncertainty modeling. By addressing these areas, GraSP is poised to further accelerate dataset creation and promote transparency and reuse in LLM development, continuously ensuring its utility across generative AI applications.
