TLDR: DeFacto is a new framework that improves multimodal language models’ (MLLMs) ability to reason faithfully with images. It uses counterfactual training, where models learn to identify relevant visual evidence, abstain from answering when evidence is masked, and avoid relying on irrelevant information. This approach, combined with reinforcement learning and a custom reward system, significantly boosts both answer accuracy and the reliability of visual reasoning across various tasks.
In the rapidly evolving landscape of artificial intelligence, multimodal language models (MLLMs) have made significant strides in understanding and reasoning with both images and text. These advanced models, often described as “thinking with images,” integrate visual information directly into their reasoning processes. However, a critical challenge persists: even when MLLMs provide correct answers, their reasoning might be flawed, relying on irrelevant parts of an image due to prior knowledge or biases in the training data. This raises concerns about the true understanding and trustworthiness of these models.
To tackle this fundamental issue, researchers have introduced a novel framework called DeFacto. This innovative approach aims to ensure that MLLMs not only provide accurate answers but also engage in faithful, evidence-grounded reasoning. DeFacto achieves this by employing a counterfactual reasoning framework that jointly enforces both correct answering and reliable reasoning.
The Core Idea: Counterfactual Thinking
DeFacto’s strength lies in its unique training methodology, which incorporates three complementary paradigms:
- Positive Supervision: The model is trained with original images and questions, learning to identify and highlight the essential visual evidence (bounding boxes) while providing the correct answer. It receives positive feedback only when both the evidence selection and the answer are accurate.
- Counterfactual Abstention: In this scenario, the crucial evidence regions in an image are intentionally masked out. The model is then expected to “abstain” from answering, typically by outputting a designated token like “unknown.” Any concrete answer in this case is penalized, forcing the model to recognize when it lacks sufficient visual information.
- Random Masking: To prevent the model from simply associating masked regions with abstention, irrelevant parts of the image are randomly masked. The model is still expected to provide the correct answer and identify the relevant evidence, ensuring it doesn’t learn superficial correlations.
This sophisticated training setup compels the model to align its reasoning trajectories with actual visual evidence, rather than relying on shortcuts or biases. It teaches the model not just where to look but also when to abstain if the necessary information is absent.
Automating Dataset Construction
A significant challenge in counterfactual reasoning is creating the necessary training data. DeFacto addresses this with an automated pipeline that constructs a dataset of approximately 100,000 images without requiring manual annotations. This pipeline involves:
- Descriptor Extraction: An MLLM (Qwen2.5-VL) analyzes the question and image to identify key visual concepts or phrases relevant to the query.
- Evidence Localization: Using an open-vocabulary detector (DINO-X) and an OCR module, candidate regions in the image are identified and categorized as either essential evidence (R+) or irrelevant regions (R-).
- Instance Generation: Based on these identified regions, the positive, counterfactual (evidence masked), and random-masking (irrelevant regions masked) instances are automatically generated.
This diverse dataset, drawing from benchmarks like VQAv2, OKVQA, and TextVQA, ensures that DeFacto’s counterfactual supervision is effective across various visual reasoning tasks, including natural images, scientific diagrams, and documents.
Reinforcement Learning for Faithful Reasoning
DeFacto further refines its models using GRPO-based reinforcement learning. This training strategy incorporates a composite reward system with three key components:
- Answer Correctness Reward: Encourages accurate answers in positive and random cases, while specifically rewarding “unknown” responses in counterfactual scenarios and penalizing unsupported guesses.
- Format Consistency Reward: Ensures the model’s outputs strictly adhere to a structured format, including bounding boxes, a thinking rationale, and the final answer.
- Region Selection Coherence Reward: Promotes the selection of evidence-bearing regions (R+) and penalizes the selection of irrelevant regions (R-), reinforcing the link between reasoning and visual evidence.
This comprehensive reward system guides the model towards producing reasoning that is logically grounded, answers that are accurate, and predictions where reasoning and outcomes are consistently faithful to the visual input.
Also Read:
- Disentangling Information for Smarter Recommendations
- Foundation Models: Guiding AI Through the Unknown in Reinforcement Learning
Demonstrated Impact
Extensive experiments on a wide array of benchmarks, including general VQA, document VQA, and scene text-centric tasks, show that DeFacto consistently achieves state-of-the-art performance. It significantly improves both answer accuracy and the faithfulness of visual grounding compared to strong baselines. For instance, it shows notable gains on VQAv2, OKVQA, and GQA, demonstrating stronger compositional and commonsense reasoning. Even on complex document and OCR-centric tasks, DeFacto establishes new benchmarks.
Ablation studies further underscore the importance of counterfactual supervision and the region-level reward design. Introducing counterfactual data with abstention alignment alone provides clear gains, and the full DeFacto framework with its tailored reinforcement learning rewards yields the best results, confirming the necessity of each component for robust and region-faithful reasoning.
DeFacto represents a significant step forward in developing more interpretable and trustworthy multimodal AI systems. By explicitly enforcing evidence-grounded reasoning and teaching models to recognize when they lack sufficient visual information, it lays a stronger foundation for future advancements in areas like video understanding and embodied AI. You can find more details about this research in the full paper: DEFACTO: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoning.
