TLDR: FaithAct is a novel framework for Multimodal Large Language Models (MLLMs) that addresses the problem of ‘unfaithfulness’ in AI reasoning. It introduces FaithEval to quantify perceptual faithfulness (how well reasoning aligns with visual input) and FaithAct to enforce it. By verifying each reasoning step against visual evidence before it’s accepted, FaithAct significantly improves perceptual faithfulness and mitigates hallucinations without compromising task accuracy, leading to more trustworthy AI explanations.
Large Language Models (LLMs) that can understand and process multiple types of data, especially text and images (known as Multimodal Large Language Models or MLLMs), have made incredible strides. However, a persistent challenge remains: unfaithfulness. This means that while these models can produce very convincing explanations, their underlying reasoning might not always be truly grounded in the visual evidence or consistent with their final conclusions. This can lead to what we call ‘hallucinations’ – where the model claims to ‘see’ objects or details that aren’t actually present in the image.
Researchers have identified two main types of unfaithfulness: behavioral faithfulness, which is about whether the model’s reasoning steps align with its final output, and perceptual faithfulness, which focuses on whether the reasoning aligns with the visual input. The new research introduces a framework called FaithAct, which prioritizes perceptual faithfulness, ensuring that every step of the model’s reasoning is firmly rooted in what is visually observable in the image.
Evaluating Faithfulness with FaithEval
To tackle this problem, the researchers first developed FaithEval, a systematic pipeline designed to quantify perceptual faithfulness. This evaluation process works in three key stages:
- Extraction of Claimed Objects: The system identifies all visual entities or attributes that the MLLM mentions in its reasoning steps. For example, if the model says, “I see a yellow bicycle,” it extracts “yellow bicycle.”
- Preference Polling and Grounding: It then checks if these claimed objects actually exist in the image. A ‘polling’ module estimates the likelihood of an object’s existence, acting as an initial filter. If an object is likely present, a ‘grounding’ module then tries to localize it within the image, providing specific visual regions and confidence scores.
- Faithfulness Scoring: Finally, these existence and localization confidences are combined to give a faithfulness score for each reasoning step and an overall score for the entire reasoning chain. A higher score means the reasoning is more consistently grounded in visual evidence.
FaithAct: A Faithfulness-First Approach to Reasoning
Building on FaithEval, the core innovation is FaithAct, a framework that integrates faithfulness directly into the reasoning process itself. Unlike traditional methods where models generate reasoning first and then might verify it, FaithAct operates on a “verify-as-you-generate” principle. This means each reasoning step is explicitly checked for visual support before it’s allowed to become part of the final reasoning chain.
FaithAct works through a planning loop with two phases: verification and action. The model, acting as a reasoning controller, uses a set of callable functions to interact with the visual evidence:
- Poll(): Checks the probability of an object’s existence.
- Ground(): Locates objects in the image with bounding boxes and confidence scores.
- Select(): Accepts an object as ‘existent’ if its confidence meets a certain threshold.
- Abstain(): Rejects an object if its confidence is too low.
- Count(): Counts the number of reliably grounded objects.
If a proposed reasoning step fails to meet the perceptual faithfulness threshold, FaithAct doesn’t just discard it. Instead, it enters a “refine-based reasoning” phase, where the MLLM is prompted to revise or regenerate the unfaithful step, using the updated, verified visual evidence. This iterative correction ensures that the reasoning remains both interpretable and grounded.
Also Read:
- MVU-Eval: A New Benchmark for AI’s Multi-Video Understanding
- SPATIALTHINKER: Advancing 3D Spatial Understanding in Multimodal AI Models
Key Findings and Impact
Experiments across various multimodal benchmarks and MLLMs (like Qwen-2.5-VL-7B, InternVL3-8B, and LLaVA-One-Vision-1.5-8B) showed significant improvements. FaithAct consistently enhanced reasoning faithfulness, with some models seeing up to a 26% improvement in perceptual faithfulness. Crucially, this improvement came without degrading the model’s accuracy in solving tasks. The framework also proved highly effective in mitigating hallucinations, especially on benchmarks designed to test for this issue.
The research highlights that treating faithfulness as a core design principle, rather than just an evaluation metric, not only reduces hallucinations but also leads to more stable and trustworthy reasoning trajectories in MLLMs. This work provides a unified framework for both evaluating and enforcing faithfulness, paving the way for more reliable and understandable AI systems. You can read the full research paper here.
