TLDR: This research paper investigates the pervasive “text bias” in Multimodal Large Language Models (MLLMs), where models prioritize textual over visual information. Contrary to previous beliefs attributing this to external factors, the authors hypothesize that the bias is intrinsic, stemming from visual key vectors being “out-of-distribution” relative to the text key space learned during language-only pretraining. Through qualitative (t-SNE) and quantitative (MMD, JS divergence) analysis of key vectors from LLaVA and Qwen2.5-VL, they found a statistically significant and persistent separation between visual and textual key subspaces. This evidence suggests that text bias arises from an inherent architectural misalignment in the attention mechanism, advocating for future research to focus on improving cross-modal key-space alignment rather than solely external data-level interventions.
Multimodal Large Language Models (MLLMs) are designed to understand and process both visual and textual information, performing well on various tasks that combine images and text. However, a significant challenge persists: these models often show a strong preference for textual inputs, frequently overlooking crucial visual evidence. This phenomenon, known as “text bias,” limits their ability to truly reason from visual information.
Traditionally, researchers have attributed this bias to external factors, such as imbalances in training data or how the models are fine-tuned with instructions. However, a new research paper titled “Unveiling Intrinsic Text Bias in Multimodal Large Language Models through Attention Key-Space Analysis” proposes a different perspective: the bias might originate from the model’s internal architecture itself. The authors of this paper are Xinhan Zheng, Huyu Wu, Xueting Wang, and Haiyun Jiang. You can read the full research paper here: Unveiling Intrinsic Text Bias in Multimodal Large Language Models through Attention Key-Space Analysis.
The Core Hypothesis: Visual Keys Are Out-of-Distribution
The central idea put forth by the researchers is that the “key vectors” generated from visual inputs (Visual Keys) are fundamentally different, or “out-of-distribution” (OOD), compared to the key vectors learned from text during the model’s initial language-only pretraining. In the attention mechanism, these key vectors are used to calculate similarity scores. If visual keys are OOD, they receive systematically lower similarity scores, leading to their under-utilization when the model forms its understanding of the context.
Investigating the Attention Key Space
To test this hypothesis, the researchers conducted both qualitative and quantitative analyses. They extracted key vectors from two popular open-source MLLMs: LLaVA-1.5-7B and Qwen2.5-VL-7B. These models were chosen because they use different vision encoders and adapter designs, offering a good basis for comparison.
For qualitative analysis, they used a technique called t-SNE (t-Distributed Stochastic Neighbor Embedding) to visualize the high-dimensional key vectors in a two-dimensional space. This allowed them to visually observe how visual and textual keys clustered.
For quantitative analysis, they employed Maximum Mean Discrepancy (MMD) and Jensen–Shannon (JS) divergence. These statistical methods help measure the distance and differences between the distributions of visual and textual key vectors. They also used “intra-modality controls” (comparing image keys to other image keys, and text keys to other text keys) as a baseline to ensure their cross-modality measurements were reliable and not just noise.
Key Findings: A Persistent Modality Gap
The results provided compelling evidence for their hypothesis. The t-SNE visualizations clearly showed that visual and textual keys occupied distinctly separate subspaces within the attention space. Visual tokens tended to form compact clusters, while textual tokens spread out, often spiraling around the visual clusters. This separation was evident across different layers of the models and various benchmarks (MMBench-CN and MMMU datasets), indicating a persistent geometric gap.
Quantitatively, the MMD and JS divergence scores confirmed a significant “modality gap.” The mean MMD for comparisons between image and text (cross-modality) was substantially higher (around 0.408) than for comparisons within the same modality (intra-modality), which were near zero (around 0.012 for image vs. image and 0.053 for text vs. text). This difference was statistically significant, ruling out measurement noise and directly supporting the idea of fundamental structural differences between visual and textual representations.
The study also highlighted differences between the models. LLaVA, which uses a simpler linear projection adapter, exhibited a larger and more robust modality bias throughout its layers, with MMD values peaking at 1.054. Qwen, with its more complex Q-Former design, managed to bring the mean features closer in deeper layers. However, its JS Divergence plot still showed a persistent gap (median around 0.45), indicating that even if the average features were closer, their overall shape and density distributions remained distinct, revealing an enduring modality bias.
Also Read:
- Advancing Visual Reasoning in AI with Latent Chain-of-Thought Models
- Video Pretraining: A New Frontier for Visual Intelligence in AI Models
Implications for Future Multimodal AI
These findings suggest that the text bias in MLLMs is not merely a consequence of external factors like data imbalance but is deeply rooted in an intrinsic misalignment within the attention key space. This shifts the focus for improving MLLMs from solely data-level fixes to addressing this fundamental architectural issue. The researchers propose that future work should concentrate on studying and improving cross-modal key-space alignment to design more balanced and truly interpretable multimodal systems.
