TLDR: This research paper investigates a new source of hallucinations in Multimodal Large Language Models (MLLMs) called ‘modality conflict,’ where inconsistencies exist between visual and textual inputs. The authors define three types of conflict (object, attribute, relationship) and introduce a new dataset, MMMC, to simulate these issues. They propose and evaluate three methods—prompt engineering, supervised fine-tuning, and reinforcement learning—to mitigate these hallucinations, finding that reinforcement learning achieves the best performance in making MLLMs more robust.
Multimodal Large Language Models (MLLMs) have shown remarkable abilities in tasks that combine vision and language, such as understanding images and responding to questions about them. However, a significant challenge these models face is hallucination, where they generate information that isn’t supported by the input data. This paper delves into a specific, often overlooked, cause of these hallucinations: modality conflict.
Understanding Modality Conflict
Unlike previous research that focused on conflicts between a model’s output and its input, this study investigates conflicts that are inherent in the inputs themselves, coming from different modalities (like an image and a text query). This puts the MLLM in a difficult position, directly leading to incorrect responses. The researchers formally define modality conflict and categorize it into three types:
- Object Conflict: This happens when the text input mentions an object that simply isn’t present in the image. For example, asking “What color is the ball?” when there’s no ball in the picture.
- Attribute Conflict: Here, the image and text describe the same object, but with different characteristics. An example would be the text mentioning a “red apple” while the image clearly shows a green one.
- Relationship Conflict: This occurs when the image and text describe the same objects, but with different relationships between them. For instance, the text might say “a cat on the table,” but the image shows the cat on the floor.
To study this phenomenon, the researchers created a new dataset called Multimodal Modality Conflict (MMMC). This dataset contains 20,000 image-question-answer sets designed to simulate these three types of conflicts. Initial evaluations on popular MLLMs like InstructBLIP, LLaVA, and Qwen2-VL-Instruct revealed that most of them struggle to recognize modality conflicts and are highly prone to hallucinations, with hallucination rates often exceeding 40%.
Strategies to Combat Hallucinations
The paper proposes three methods to reduce hallucinations caused by modality conflict:
- Prompt Engineering: This is the simplest approach, involving a special instruction added to the user’s question. For example, the prompt might ask the model to “Please check if the image contains mentioned information and answer the question.” This method is easy to implement and requires no extra data or computing power, but its effectiveness can vary depending on the base model.
- Supervised Fine-Tuning (SFT): This method involves training the MLLM on the MMMC dataset. By optimizing the model to align its predictions with the correct answers in the dataset, SFT aims to improve the model’s robustness. While more effective than prompt engineering, it requires additional data and computational resources.
- Reinforcement Learning (RL): Inspired by its success in aligning models with human preferences, this method optimizes the MLLM by rewarding it for generating semantically consistent answers and penalizing hallucinated responses. RL allows the model to explore more diverse and robust answers during training. The study found that this method achieved the best performance in mitigating hallucinations under modality conflict, although it can be less stable during training compared to SFT.
Also Read:
- Reinforcement Fine-tuning: A Robust Approach to Continual Learning in Large Language Models
- Unpacking Efficiency in Multimodal AI: Addressing Redundant Vision Encoders
Key Findings and Implications
Experiments on the MMMC dataset showed that all three proposed methods significantly improved the robustness of MLLMs, reducing hallucination rates by 10% to 50%. Reinforcement learning generally yielded the best results, while supervised fine-tuning also showed strong and consistent performance. The study also highlighted that MLLMs perform better on object-conflict types, moderately on attribute-conflict scenarios, and struggle most with relationship-conflict types, indicating a need for further improvement in handling complex relational dynamics.
This research sheds light on an important, often overlooked, source of hallucinations in MLLMs and provides valuable insights into making these models more robust and reliable in real-world applications. The code and dataset are publicly available for further research at https://github.com/zmzhang2000/MMMC.
