TLDR: DP2O-SR is a new framework for real-world image super-resolution that improves image quality by directly optimizing generative models based on perceptual preferences. It uses a hybrid reward system combining full-reference and no-reference image quality assessment, and a novel method for creating preference pairs from a single model’s diverse outputs. The framework also introduces Hierarchical Preference Optimization to adaptively weight training signals, leading to significant improvements in perceptual quality, generalization, and output stability without requiring human annotations.
Image Super-Resolution (ISR) is a fascinating field focused on transforming blurry, low-resolution images into sharp, high-resolution masterpieces. Traditionally, methods aimed for pixel-perfect accuracy, but this often resulted in images that looked unnaturally smooth, lacking the rich textures we see in real life. The focus has since shifted to ‘perceptual quality’ – making images look realistic and pleasing to the human eye, especially for real-world scenarios where original image degradations are complex and unknown.
Recent advancements in generative models, particularly large-scale text-to-image (T2I) diffusion models like Stable Diffusion and FLUX, have shown immense promise in Real-ISR. These models can synthesize incredibly plausible and diverse details. However, they come with a catch: their inherent randomness. Different noise inputs can lead to outputs with varying perceptual quality, a characteristic often seen as a limitation. But what if this randomness could be harnessed as a strength?
Introducing DP2O-SR: Optimizing for Perceptual Excellence
A new framework, Direct Perceptual Preference Optimization for Real-World Image Super-Resolution, or DP2O-SR, proposes to do just that. It aims to align generative ISR models with human-like perceptual preferences without the need for expensive human annotations. Instead, DP2O-SR leverages the inherent variability of T2I models, treating the range of possible outputs as a source of valuable supervision.
The core of DP2O-SR lies in its innovative perceptual reward system. This system combines two types of image quality assessment (IQA) models: full-reference (FR) metrics, which compare an output against a perfect original image to ensure structural fidelity, and no-reference (NR) metrics, which evaluate quality without an original reference, focusing on natural appearance and aesthetic coherence. By blending these, DP2O-SR creates a balanced reward signal that encourages both accuracy and naturalness. For instance, using only FR metrics might lead to overly smooth images, while relying solely on NR metrics could result in unrealistic ‘hallucinations.’ The hybrid approach ensures rich, natural details while maintaining structural consistency.
Smart Preference Data Curation
Unlike previous methods that might pick a ‘best’ and ‘worst’ image from different models, DP2O-SR takes a more nuanced approach. It samples multiple outputs from a *single* model using different random noise seeds. These outputs are then ranked by the perceptual reward, and numerous preference pairs are constructed from the top-performing and bottom-performing samples. This method provides a richer training signal, capturing finer perceptual distinctions and making better use of the diversity generated by the model.
The researchers also explored how the number of samples and the selection ratio (how many top/bottom samples are chosen) impact learning. They found that larger models benefit from stronger contrast in supervision (fewer top/bottom samples), while smaller models perform better with broader coverage (more top/bottom samples) to ensure stable learning gradients. This highlights the importance of tailoring data curation strategies to the specific model’s capacity.
Hierarchical Preference Optimization (HPO)
To further refine the learning process, DP2O-SR introduces Hierarchical Preference Optimization (HPO). This technique adaptively weights training pairs, recognizing that not all comparisons are equally informative. HPO operates at two levels: ‘intra-group’ weighting prioritizes comparisons with larger reward differences within the same set of generated images, while ‘inter-group’ weighting focuses on input images that yield a greater spread of perceptual quality in their generated outputs. By emphasizing the most informative signals, HPO makes training more efficient and stable.
Also Read:
- Enhancing AI Safety: A Training-Free Approach to Detecting Out-of-Distribution Data
- SSL4RL: Guiding AI to Deeper Visual Understanding with Self-Supervised Rewards
Impressive Results and Generalization
Extensive experiments demonstrated that DP2O-SR significantly improves perceptual quality across various generative backbones, including both diffusion- and flow-based T2I models. It consistently outperformed baseline models and a wide range of state-of-the-art Real-ISR methods on challenging real-world benchmarks. The improvements were seen not only in metrics used during training but also in untrained perceptual metrics, indicating strong generalization capabilities.
Qualitative comparisons visually confirm these improvements. DP2O-SR effectively removes artifacts, reconstructs fine details like text and architectural patterns, and generates more semantically faithful images compared to other methods. Interestingly, even though the reward function assesses overall image quality, DP2O-SR often leads to localized refinements, such as sharper wing textures, while leaving other regions unchanged. This suggests the model implicitly learns to prioritize perceptually important areas.
Furthermore, DP2O-SR enhances the stability of generative models. By improving the ‘worst-case’ outputs, it leads to more consistent and perceptually robust results, reducing the variability in quality that can arise from the models’ stochastic nature.
While DP2O-SR marks a significant step forward in Real-ISR, the authors acknowledge limitations, such as the interpretability of IQA-based rewards and the current offline training pipeline. Future work will explore more accurate reward models and iterative optimization. For more details, you can read the full research paper here.
