TLDR: GHOST is an automated method that generates subtle, hallucination-inducing images to stress-test Multimodal Large Language Models (MLLMs). It optimizes image embeddings to mislead MLLMs into perceiving objects that aren’t present, then uses a diffusion model to create natural-looking images. GHOST achieves high hallucination success rates, uncovers transferable vulnerabilities across models, and can even be used for fine-tuning to mitigate hallucinations, making it a diagnostic and corrective tool for MLLM reliability.
Multimodal Large Language Models (MLLMs) have made incredible strides in understanding and generating content across both vision and language. From describing images to answering complex visual questions, their capabilities seem vast. However, these advanced AI systems aren’t without their flaws. One significant and concerning issue is ‘object hallucination,’ where an MLLM incorrectly perceives an object in an image that simply isn’t there.
Traditionally, researchers have studied this weakness using static benchmarks – fixed sets of images and scenarios. While useful, these methods can only reveal known vulnerabilities and might miss unique failure modes specific to different MLLMs. To address this, a new method called GHOST (Generating Hallucinations via Optimizing Stealth Tokens) has been introduced. GHOST is designed to actively stress-test MLLMs by creating images that are specifically crafted to induce these hallucinations.
How GHOST Works Its Magic
Imagine an image of a banana on a plate, with no knife in sight. An MLLM would correctly identify the absence of a knife. GHOST takes this original image and subtly modifies it. It doesn’t add a visible knife, but introduces tiny, semantic cues – perhaps altering the banana’s stem to vaguely resemble a knife’s edge. To a human, the knife is still clearly absent, but the MLLM is tricked into ‘seeing’ it.
The process is fully automated and requires no human supervision. GHOST operates by optimizing in the image embedding space. This means it tweaks the numerical representation of the image in a way that misleads the MLLM, all while ensuring the target object remains visually absent. This optimized embedding then guides a diffusion model to generate a natural-looking image that incorporates these subtle, misleading cues.
A key innovation of GHOST is its ability to decouple the optimization process from the image generator. Previous methods often required complex, resource-intensive pipelines. GHOST introduces a ‘mapper’ that aligns the visual spaces of the target MLLM and the diffusion model, making the process much more efficient and adaptable across different AI models.
Impressive Results and Transferable Vulnerabilities
The effectiveness of GHOST is striking. When tested on models like Qwen2.5-VL, it achieved a hallucination success rate exceeding 28%, a significant leap compared to around 1% in prior data-driven discovery methods. The generated images are not only high-quality but also confirmed to be object-free from a human perspective, with human evaluators agreeing 89% of the time that the target object was absent.
Perhaps even more critically, GHOST uncovers ‘transferable vulnerabilities.’ This means that images optimized to induce hallucinations in one MLLM often cause similar hallucinations in other, different models. For instance, images generated for Qwen2.5-VL induced hallucinations in GPT-4o at an impressive 66.5% rate. This suggests that different MLLMs share common blind spots and biases, highlighting systemic issues rather than isolated incidents.
Also Read:
- Decoding How AI Understands the World: A Multimodal Perspective
- How Narrative Attacks Exploit Unified AI Models
Beyond Diagnosis: A Corrective Tool
GHOST isn’t just a diagnostic tool; it also offers a path to mitigation. The research demonstrates that fine-tuning MLLMs on these GHOST-generated, hallucination-inducing images can improve model robustness on downstream hallucination benchmarks. This positions GHOST as a valuable tool for building more reliable and trustworthy multimodal AI systems.
In conclusion, GHOST represents a significant step forward in understanding and addressing object hallucination in MLLMs. By systematically generating images that expose these flaws, it provides both a powerful diagnostic method and a potential corrective mechanism for enhancing the reliability of next-generation AI. For more technical details, you can read the full research paper here.
