TLDR: SplitMeanFlow is a new framework for training few-step generative models that significantly reduces computational cost. Unlike previous methods that rely on complex differential equations, SplitMeanFlow uses a simple algebraic identity based on the additivity of integrals. This approach eliminates the need for computationally expensive calculations, leading to more stable training and broader hardware compatibility. The method has been successfully deployed in large-scale speech synthesis products, achieving up to a 20x speedup in generation while maintaining high quality, enabling one-step generation comparable to multi-step baselines.
Generative models, which are powerful tools for creating realistic images, videos, and audio, have made incredible strides. However, their practical use is often limited by a significant hurdle: they typically require many computational steps to generate high-quality outputs. This iterative process can be very slow, especially in situations where quick responses are needed, like in real-time applications.
To tackle this challenge, researchers have been focusing on developing “few-step” or even “one-step” generative models. One prominent method in this area is MeanFlow, which introduced the concept of learning the “average velocity” of the transformation from noise to data, rather than the “instantaneous velocity” at each moment. While effective, MeanFlow relies on a complex mathematical concept called a “differential identity,” which involves derivatives and can be computationally demanding.
Introducing SplitMeanFlow: A Simpler, More Powerful Approach
A new research paper, titled “SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling,” proposes a fresh perspective. Authored by Yi Guo, Wei Wang, Zhihang Yuan, Rong Cao, Kuan Chen, Zhengyang Chen, Yuanyuan Huo, Yang Zhang, Yuping Wang, Shouda Liu, and Yuxuan Wang, this work argues that MeanFlow’s differential approach is a specific instance of a more fundamental principle. The authors return to the basic definition of average velocity and leverage a simple, yet powerful, property of integrals: additivity.
Imagine a journey. The total distance you travel is simply the sum of the distances covered in smaller segments of that journey. SplitMeanFlow applies this intuitive idea to the “flow” of data generation. This leads to a novel, purely algebraic identity called “Interval Splitting Consistency.” This identity describes how average velocities relate across different time intervals without needing any complex differential equations.
The core of SplitMeanFlow is to directly enforce this algebraic consistency as a learning objective during training. This means the model essentially “supervises itself” by ensuring that the average velocity over a large interval is consistent with the sum of average velocities over its sub-intervals.
Why SplitMeanFlow is a Step Forward
The paper highlights two key advantages of SplitMeanFlow:
-
Theoretical Generality: The researchers formally prove that MeanFlow’s differential identity is actually a special, limited case of SplitMeanFlow’s algebraic consistency. This means SplitMeanFlow provides a more comprehensive and robust foundation for learning average velocity fields.
-
Practical Efficiency: Perhaps the most significant practical benefit is that SplitMeanFlow eliminates the need for Jacobian-Vector Product (JVP) computations during training. JVP calculations are often complex, computationally expensive, and can lead to training instability. By removing this requirement, SplitMeanFlow offers a simpler implementation, more stable training, and broader compatibility with various hardware.
Also Read:
- SADA: Enhancing Diffusion Model Efficiency Through Adaptive Acceleration
- Efficient Discrete Data Synthesis: Introducing Rectified Discrete Flow
Real-World Impact in Speech Synthesis
The effectiveness of SplitMeanFlow isn’t just theoretical; it has been successfully deployed in large-scale industrial products, specifically ByteDance’s Doubao speech synthesis system. This real-world application demonstrates its practical value and robustness.
Experiments in audio generation tasks show compelling results. A 2-step SplitMeanFlow model achieved performance comparable to a 10-step Flow Matching baseline, significantly reducing computational steps while maintaining high quality. Even more impressively, a 1-step SplitMeanFlow model achieved performance on par with the 10-step Flow Matching baseline across all evaluation metrics. This represents a remarkable 20x reduction in computational cost without any noticeable loss in audio quality, fulfilling a major goal in efficient generative modeling.
In conclusion, SplitMeanFlow offers a novel and principled framework for training few-step generative models. By leveraging the fundamental additivity of integrals to derive an algebraic consistency identity, it provides a more general, efficient, and stable approach to learning average velocity fields. This advancement opens promising new avenues for developing even more powerful and efficient generative models in the future. You can read the full research paper here: SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling.
