TLDR: Refine-IQA is a new multi-stage reinforcement fine-tuning (RFT) framework for Image Quality Assessment (IQA) in large multi-modal models (LMMs). It addresses limitations of previous methods by first enhancing the model’s fundamental visual perception using a new dataset (Refine-Perception-20K) and multi-task rewards (Stage 1). Then, it introduces a ‘probability difference reward’ to effectively supervise and prevent the ‘think collapse’ phenomenon in the model’s internal reasoning process (Stage 2). This results in LMMs that not only accurately score image quality but also develop a robust ‘think’ capability for detailed quality interpretation, rivaling larger models with less training data.
In the rapidly evolving field of artificial intelligence, assessing the quality of images is a crucial task, known as Image Quality Assessment (IQA). This is particularly important for large multi-modal models (LMMs) that process both visual and textual information. While traditional methods often rely on supervised fine-tuning (SFT), which can lead to issues like overfitting, a new approach called reinforcement fine-tuning (RFT) is gaining traction. However, existing RFT methods for IQA have faced challenges, especially in effectively guiding the model’s internal ‘think’ process and truly enhancing its fundamental visual perception capabilities.
A recent research paper, Refine-IQA: Multi-Stage Reinforcement Finetuning For Perceptual Image Quality Assessment, introduces an innovative multi-stage RFT framework designed to overcome these limitations. The core idea is to not only improve how LMMs score image quality but also to cultivate a robust ‘think’ capability that allows them to interpret and reason about visual quality, much like humans do.
Stage 1: Building Foundational Visual Perception
The first stage of Refine-IQA focuses on strengthening the model’s inherent ability to perceive low-level visual quality features. To achieve this, the researchers created a unique dataset called Refine-Perception-20K. This dataset is meticulously designed for RFT and contains over 20,000 images with 12 major distortion categories and 5 levels of severity. These distortions are localized, meaning they affect specific objects within an image, rather than the entire picture. The dataset was built through a rigorous process involving automated filtering and human-in-the-loop scrutiny to ensure high quality and semantic consistency.
During this stage, the model is trained with a multi-task reward system. This system encourages the model to perform three key tasks: recognizing the type and severity of visual distortions, identifying the specific semantic object that is distorted, and accurately pinpointing the area of distortion using bounding box coordinates. This comprehensive training helps the LMM become more sensitive to subtle visual imperfections, laying a strong foundation for more accurate quality assessment.
Stage 2: Cultivating the ‘Think’ Process with Probability Difference Reward
The second stage addresses a critical issue observed in previous RFT-based IQA models: the ‘think collapse’ phenomenon. In the ‘think-answer’ paradigm, LMMs are expected to first ‘think’ (generate an interpretation and reasoning about image quality) and then ‘answer’ (provide a quantitative quality score). However, without proper supervision, the ‘think’ process often deteriorates, becoming minimal or irrelevant to the final decision.
Refine-IQA tackles this by introducing a novel ‘probability difference (PD) reward’. This reward implicitly supervises the ‘think’ process by measuring the difference in the likelihood of generating the correct answer’s tokens between the ‘think’ mode and a ‘no-think’ mode. Essentially, it incentivizes the model to produce a more meaningful and contributing ‘think’ process that genuinely helps in arriving at the correct quality score. This ensures that the model’s internal reasoning is not just a byproduct but an active and effective component of its decision-making.
Outstanding Performance and Robust Interpretation
The Refine-IQA Series Models, developed using this multi-stage approach, have demonstrated impressive results. They achieve outstanding performance on both image quality perception tasks (identifying distortions, objects, and areas) and image quality scoring tasks across various datasets. Notably, the models show a negligible performance gap between ‘think’ and ‘no-think’ modes during evaluation, indicating the effectiveness of the cultivated ‘think’ process.
Furthermore, the research highlights that this paradigm activates a robust ‘think’ (quality-interpreting) capability. Even with a relatively small dataset for quality scoring RFT (approximately 13K samples), the Refine-IQA models perform competitively with large-scale SFT-based LMMs on quality interpretation benchmarks. This suggests that the ‘think’ process, refined through Stage 2, significantly enhances the model’s ability to explain and interpret visual quality.
Also Read:
- Addressing AI Hallucinations in Multimodal Models Through Causal Understanding
- Chain of Questions: Empowering Language Models with Multimodal Curiosity
Looking Ahead
The Refine-IQA framework offers compelling insights for developing efficient and robust visual quality assessment agents. By focusing on enhancing fundamental visual perception and explicitly incentivizing a meaningful ‘think’ process, this work paves the way for LMMs that are not only accurate in their assessments but also capable of providing detailed, human-like interpretations of image quality.
