TLDR: RealDPO is a novel method that uses real-world videos as preferred examples to train video generative models, improving the realism of complex motions, especially human actions. It avoids the limitations of reward models by directly contrasting real data with generated outputs and introduces a new dataset, RealAction-5K, for this purpose. Experiments show it significantly enhances video quality, text alignment, and motion realism.
Video generative models have made remarkable strides in creating realistic visual content. However, a persistent challenge remains: generating complex and natural motions, especially those involving human activities. Often, existing models produce movements that appear unnatural, lacking smoothness and contextual consistency. This gap between what models generate and real-world motion limits their practical use.
A new research paper titled “REALDPO: REAL ORNOTREAL,THAT IS THEPREFERENCE” by Guo Cheng, Danni Yang, Ziqi Huang, Jianlou Si, Chenyang Si, and Ziwei Liu introduces an innovative solution called RealDPO. This novel alignment method tackles the issue by leveraging real-world data as positive examples for preference learning, leading to more accurate and lifelike motion synthesis.
Traditional methods for improving video generation often rely on what are called “reward models.” These models score synthetic data, and the generative model learns from these scores. However, this approach has several drawbacks, including “reward hacking” (where the model optimizes for the score rather than true quality), scalability issues for high-resolution videos, and potential biases. RealDPO bypasses these limitations by eliminating the need for an external reward function altogether.
Instead of a reward model, RealDPO employs Direct Preference Optimization (DPO), a technique that uses pairs of preferred (win) and non-preferred (lose) samples to guide the learning process. In RealDPO’s unique approach, high-quality real-world videos serve as the “win” samples, while the model’s own erroneous outputs become the “lose” samples. By contrasting these, RealDPO enables the model to iteratively self-correct and progressively refine its motion quality.
To facilitate this training, the researchers also introduce RealAction-5K, a meticulously curated dataset of high-quality videos. This dataset captures a diverse range of human daily activities with rich and precise motion details. It emphasizes a “less is more” principle, demonstrating that RealDPO can achieve significant improvements with fewer high-quality real samples when combined with model-generated negative samples, unlike traditional supervised fine-tuning which typically requires more data.
The RealDPO framework is designed with a tailored DPO loss function specifically for diffusion-based transformer architectures, ensuring efficient and effective preference alignment. This design also helps in drastically reducing computational overhead by avoiding the need to decode latent to pixel space during training.
Extensive experiments have shown that RealDPO significantly enhances video quality, improves text alignment (how well the video matches its descriptive text), and boosts motion realism across various human action scenarios. It outperforms state-of-the-art models and existing preference optimization techniques, demonstrating more natural motion generation and greater stability in visual output, avoiding issues like unnatural actions or visual collapse seen in other methods.
Also Read:
- SceneAdapt: Integrating Scene Understanding into Motion Generation
- CanvasMAR: A Novel Approach for Enhanced Video Generation with Global Frame Prediction
This work represents a significant step forward in video generation, offering a robust and data-efficient framework for creating more realistic and contextually consistent motions. For more in-depth information, you can read the full research paper here.
