TLDR: Self-Forcing++ is a new method that enables diffusion models to generate high-quality videos up to 4 minutes and 15 seconds long, a significant improvement over previous models limited to short clips. It achieves this by having a “student” model learn from a “teacher” model to correct errors in its own self-generated long videos, without needing extensive long-video training data. The approach uses techniques like backward noise initialization, extended distribution matching, and a rolling KV cache to maintain temporal consistency and visual fidelity over extended durations. It also introduces a new metric, Visual Stability, for more accurate evaluation of long videos.
The world of artificial intelligence has seen incredible strides in generating realistic images and videos, largely thanks to advanced diffusion models. However, a persistent challenge has been the ability to create high-quality, long-duration videos. Most state-of-the-art models are typically limited to generating short clips, often just 5 to 10 seconds long. This limitation stems from the complex architectures and high computational costs involved in extending generation to longer sequences.
A new research paper introduces a groundbreaking approach called Self-Forcing++ that aims to overcome this hurdle. Authored by Justin Cui, Jie Wu, Ming Li, Tao Yang, Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, and Cho-Jui Hsieh from institutions like UCLA, ByteDance Seed, and the University of Central Florida, this method allows for the creation of high-quality videos lasting several minutes.
The Challenge of Long Video Generation
Traditional diffusion models, especially those based on transformer architectures, face significant computational demands when generating videos. While autoregressive models have shown promise for longer videos, they often suffer from quality degradation. This happens because ‘teacher’ models, which guide the generation process, are usually trained only on short videos. When student models try to extrapolate beyond this short training horizon, errors can accumulate in the continuous latent space, leading to issues like over-exposure, flickering, or even static content.
Self-Forcing++: A Novel Solution
Self-Forcing++ proposes a simple yet highly effective way to mitigate this quality degradation without needing to train on vast datasets of long videos or requiring supervision from long-video teachers. The core idea is to leverage the rich knowledge of existing teacher models to guide a student model. This guidance is provided through sampled segments drawn from the student’s own self-generated long videos.
The method introduces several key strategies:
-
Backward Noise Initialization: This technique helps maintain temporal consistency across long videos. Instead of starting each segment from random noise, noise is re-injected into already denoised latent vectors. This ensures that the generated segments retain the temporal dependencies of previous frames.
-
Extended Distribution Matching Distillation (DMD): While teacher models are trained on short (e.g., 5-second) clips, Self-Forcing++ instructs the student model to generate much longer sequences. It then uniformly samples short, contiguous windows from these long sequences. The student is trained to minimize the difference between its distribution and the teacher’s distribution within these windows, effectively extending its learning horizon far beyond the teacher’s original limit.
-
Training with Rolling KV Cache: This crucial component eliminates a common mismatch between training and inference. By using a rolling Key-Value (KV) cache during both training and inference, the model can generate sequences far beyond the teacher’s supervisory horizon, simplifying the process and avoiding issues like recomputing overlapping frames or over-exposure.
-
Improving Long-Term Smoothness via GRPO: To combat temporal inconsistencies like objects abruptly appearing or vanishing, Self-Forcing++ incorporates Group Relative Policy Optimization (GRPO), a reinforcement learning technique. This helps promote smoother temporal transitions, leading to better long-range consistency and overall perceptual quality.
Impressive Results and a New Metric
The experimental results are remarkable. Self-Forcing++ demonstrates the capability to generate videos up to 4 minutes and 15 seconds long, which is a 50x improvement over baseline models. This is achieved while maintaining high visual quality and temporal consistency, avoiding common pitfalls like motion collapse or fidelity degradation seen in other methods.
The researchers also identified biases in existing benchmarks like VBench, which sometimes favored over-exposed or degraded videos. To address this, they propose a new metric called Visual Stability. This metric uses a state-of-the-art video MLLM (Gemini-2.5-Pro) to systematically evaluate quality degradation and over-exposure in long videos, providing a more reliable assessment.
Also Read:
- KineMask: A New Approach for Physically Plausible Video Dynamics
- SiD-DiT: Bridging Diffusion and Flow Matching for Faster Image Synthesis
Looking Ahead
Self-Forcing++ represents a significant step forward in long-video generation, paving the way for more robust and scalable video synthesis models without the need for massive real-world long-video datasets. The paper can be found here: Self-Forcing++: Towards Minute-Scale High-Quality Video Generation.
