TLDR: A new research paper introduces SUB, a synthetic image dataset, and Tied Diffusion Guidance (TDG), a novel image generation method, to benchmark the generalization capabilities of Concept Bottleneck Models (CBMs). The study reveals that CBMs and Vision Language Models (VLMs) struggle to reliably identify concepts under distribution shifts, often memorizing class-specific concept vectors rather than truly grounding predictions in image attributes. This highlights concerns about their interpretability and calls for more robust concept-based AI.
Artificial intelligence models, especially those based on deep learning, have achieved remarkable success in complex tasks. However, their lack of transparency often hinders their deployment in critical real-world applications, such as medicine, where understanding the model’s reasoning is crucial. Concept Bottleneck Models (CBMs) were introduced to address this by generating intermediate, interpretable concepts that inform the final prediction, aiming to make AI more transparent.
Despite their initial promise, recent research indicates that CBMs face significant challenges in reliably identifying the correct concepts when faced with variations in data, known as distribution shifts. This suggests that CBMs might be memorizing concept vectors associated with specific classes rather than genuinely learning to identify concepts based on visual cues in the image. For instance, a CBM trained on Blue Jays might still predict a “blue crown” even when presented with a Blue Jay image where the crown has been clearly changed to yellow. This behavior raises questions about the validity of their concept predictions as reliable interpretability tools.
To rigorously evaluate the robustness and generalization of CBMs to concept variations, a new benchmark dataset called SUB (Substitutions on Caltech-UCSD Birds-200-2011) has been introduced. This fine-grained image and concept benchmark comprises 38,400 synthetic images. It is built upon a subset of the widely used CUB dataset, focusing on 33 bird classes and 45 concepts, such as wing color or belly pattern. The core idea behind SUB is to generate images where a specific concept attribute is substituted, allowing researchers to assess how well CBMs adapt to these novel combinations of known concepts.
A key innovation enabling the creation of SUB is a novel method called Tied Diffusion Guidance (TDG). Traditional text-to-image models often struggle with precise attribute substitutions; simply prompting a model for a “Blue Jay with a yellow crown” might not yield the desired result. TDG addresses this by precisely controlling the generated images. It works by tying two parallel denoising processes, ensuring that both the correct bird class and the correct substituted attribute are generated accurately. This test-time adaptation enhances attribute-level control in text-to-image models, making it possible to create the fine-grained edits needed for the SUB dataset.
The creation of the SUB dataset involved a meticulous filtering process to ensure the quality and faithfulness of the manipulated attributes. This included automatic visual-question-answering (VQA) evaluations and human validation steps to identify and remove images that did not correctly modify the target attribute or deviated too much from the reference bird. This rigorous approach ensures that SUB provides a reliable environment for evaluating interpretable models.
Experiments using SUB revealed significant findings: CBMs and even leading Vision Language Models (VLMs) like CLIP, SigLIP, and EVA-CLIP, struggle to generalize to novel combinations of known concepts. Their ability to detect substituted attributes (S+) was often below random chance, despite high accuracy on standard training datasets. This strongly suggests that these models infer concepts from the predicted class rather than truly grounding them in the visual information of the image. For instance, some models showed a tendency to incorrectly identify the original attribute even when it was no longer present, indicating a form of hallucination.
Also Read:
- MoHoBench: A New Benchmark for Evaluating Honesty in Multimodal AI
- Unmasking Deepfakes: The Power of Visual Language Models
The implications of these findings are substantial. They raise serious concerns about the true interpretability of current CBMs and VLMs, suggesting that their performance on training classes can be misleading. The research highlights the need for developing more robust concept-based models whose predictions are genuinely grounded in the target concepts. The SUB dataset and the TDG method offer valuable tools for future research in this direction, paving the way for the next generation of CBMs with more reliable and well-grounded explanations. For more details, you can refer to the full research paper: Benchmarking CBM Generalization via Synthetic Attribute Substitutions.
