TLDR: A reproducibility study investigated FairCLIP, a method designed to reduce biases in AI vision-language models for medical applications. While the study confirmed initial biases in CLIP, it found that FairCLIP, both in its original and an aligned implementation, did not consistently improve fairness or performance in zero-shot glaucoma classification on the Harvard-FairVLMed dataset. The research also found that minimizing the Sinkhorn distance, a core mechanism of FairCLIP, did not directly lead to better fairness or performance, and its generalizability to other datasets was not evident.
In the rapidly evolving world of artificial intelligence, ensuring fairness in machine learning models is becoming increasingly critical, especially when these technologies are applied to sensitive areas like healthcare. Vision-language (VL) models, which combine visual and textual information, are gaining traction in the medical field, but they are also known to carry inherent biases. Addressing these biases is crucial, as they can directly impact patient health and outcomes.
A recent study, titled On the Reproducibility of “FairCLIP: Harnessing Fairness in Vision-Language Learning”, delved into the reproducibility of a method called FairCLIP, originally proposed by Luo et al. (2024). FairCLIP was designed to enhance the fairness of CLIP, a popular vision-language model, by reducing disparities in image-text similarity scores across different sensitive groups, such as race or gender. The core idea behind FairCLIP is to use a mathematical technique called Sinkhorn distance to minimize these score differences.
Investigating FairCLIP’s Claims
The researchers, Hua Chang Bakker, Angela Madelon Bernardy, Stan Deutekom, and Stan Fris from the University of Amsterdam, set out to reproduce the experiments from the original FairCLIP paper. Their primary goal was to verify two key claims: first, that CLIP exhibits significant biases towards specific demographics (Asian, male, non-Hispanic, and Spanish-speaking individuals) when used with the Harvard-FairVLMed dataset, and that fine-tuning CLIP can alleviate these biases, improving both fairness and performance. Second, they aimed to confirm that fine-tuning CLIP with the FairCLIP objective on this dataset improves both the performance and fairness of zero-shot glaucoma classification across various subgroups.
The Methodology: A Closer Look
The study utilized two main datasets: Harvard-FairVLMed, which contains medical images, clinical notes, and demographic attributes, and FairFace, a dataset of face images balanced across race and gender, used to test the generalizability of FairCLIP. A critical finding during the reproduction process was that the mathematical description of the FairCLIP regularizer in the original paper differed from its actual implementation. This led the researchers to create a new, aligned implementation called A-FairCLIP to accurately test the model as described. They also noted that the original model selection was performed on the test set, which is generally considered an improper practice in machine learning research.
Key Findings: Biases Confirmed, Fairness Improvements Unclear
The study’s linear probing experiments largely confirmed the first claim: CLIP does indeed show biases towards certain demographic groups in its visual features. Fine-tuning CLIP (referred to as CLIP-FT) did show some improvements in performance metrics like AUC (Area Under the Receiver Operating Characteristic Curve) and ES-AUC (Equity-Scaled AUC) for attributes like race, gender, and ethnicity. However, the fairness metrics (DPD and DEOdds) didn’t always improve, especially for more balanced attributes, and the results often had high standard deviations, suggesting instability.
However, the second claim regarding FairCLIP’s ability to improve both performance and fairness in zero-shot glaucoma classification was not supported by the experimental results. When using the official FairCLIP code, the model generally performed worse than standard fine-tuned CLIP (CLIP-FT) across most metrics. Even with the aligned implementation (A-FairCLIP), which corrected the discrepancies between the paper’s description and code, the improvements over CLIP-FT were not significant, and in some cases, performance was even worse.
The researchers observed that while FairCLIP effectively reduced the Sinkhorn distances between group distributions and the overall population distribution – meaning it was indeed trying to make the distributions more similar – this reduction did not translate into better fairness or performance in glaucoma prediction. This suggests that simply minimizing this distance might not be sufficient to achieve the desired fairness and performance outcomes.
FairCLIP+ and Generalizability
The study also examined FairCLIP+, an extension designed to handle multiple sensitive attributes simultaneously. Similar to FairCLIP, FairCLIP+ did not show significant improvements in performance or fairness. Furthermore, testing on the FairFace dataset for zero-shot gender prediction also yielded no clear performance increases, indicating that the FairCLIP objective might not generalize well to other datasets or attributes.
Also Read:
- Machine Unlearning: A Targeted Approach to Eradicating Bias in AI Vision Models
- AI Models Systematically Amplify Gender Stereotypes, But Fairness Is Within Reach
Conclusion: The Importance of Reproducibility
In summary, while the initial biases in CLIP were confirmed, the reproducibility study found that FairCLIP, in both its original and aligned forms, did not consistently improve the performance or fairness of zero-shot glaucoma classification. The findings highlight that minimizing the Sinkhorn distance, a core component of FairCLIP, does not necessarily lead to improved fairness or performance in the tested scenarios. This research underscores the critical importance of reproducibility studies in validating AI models and their claims, especially in high-stakes applications like medicine.
