TLDR: A recent study published on ResearchGate highlights widespread demographic inaccuracies and biases in patient depictions generated by leading AI text-to-image models, Bing Image Generator and Meta Imagine. The research found an over-representation of White and normal-weight individuals, and often failed to accurately reflect disease-specific demographics, raising concerns about AI’s role in amplifying healthcare misconceptions.
Artificial intelligence (AI) text-to-image generators, despite their increasing sophistication and photorealistic outputs, are perpetuating significant demographic inaccuracies and biases in their depiction of patients, according to a comprehensive study posted on ResearchGate on July 25, 2024. The research, led by Tim Wiegand and a team of authors from Ludwig-Maximilians-Universität in Munich and Broad Institute of MIT and Harvard, raises critical concerns about the potential for AI to amplify misconceptions within healthcare.
The study evaluated the demographic accuracy and potential biases in patient images generated by two widely used AI models: Microsoft’s Bing Image Generator (powered by DALL-E3) and Meta Imagine. Researchers generated a total of 4,580 images of patients across 29 different diseases, including 14 with distinct epidemiological characteristics and 15 commonly stigmatized conditions. Eight independent M.D. Ph.D. researchers meticulously rated the sex, age, weight group, and race/ethnicity of the depicted patients, comparing these against real-world epidemiological data.
The findings revealed a pervasive lack of accuracy in representing disease-specific demographic characteristics. For instance, accurate representation of age, sex, and race/ethnicity was achieved only once across all generated images – specifically for patients with multiple sclerosis depicted by Meta. In stark examples of inaccuracy, Meta’s generator depicted both male and female patients for conditions like prostate cancer (which exclusively affects males) and eclampsia (which exclusively affects females). Bing’s generator also showed substantial inaccuracies, particularly concerning the accurate representation of various races and ethnicities.
Beyond disease-specific inaccuracies, the study identified significant general biases. There was a notable over-representation of White individuals in the generated images, accounting for 68% in Bing’s output and 28% in Meta’s, compared to a pooled real-world patient data average of just 20%. Conversely, individuals of Asian, Black or African American (BAA), Hispanic or Latino (HL), Native Hawaiian or Other Pacific Islander (NHPI), and American Indian or Alaska Native (AIAN) descent were consistently under-represented. Similarly, normal-weight individuals were over-represented (88% in Bing, 93% in Meta, versus 63% in the general population), leading to a significant under-representation of overweight individuals (5% in Bing, 4% in Meta, compared to 32% in the general population). Interestingly, no substantial over-representation of male sex was observed overall.
Further analysis into stigmatized diseases uncovered additional biases. Females were consistently depicted as younger than males in images from both generators. Moreover, females were rated as having higher weight than males. Racial and ethnic biases also emerged, with Asian, BAA, HL, NHPI, and AIAN individuals combined being portrayed as having more weight than White individuals, an observation that contradicts real-world demographic trends.
The researchers attribute these inaccuracies primarily to the composition of the AI models’ training data. These models are typically trained on vast, non-medical datasets from the internet, which often lack sufficient images of actual patients and crucial epidemiological information. While bias mitigation strategies are employed, the study suggests these can sometimes lead to over-correction, as seen in the depiction of both sexes for sex-specific diseases.
Also Read:
- Study Reveals ChatGPT’s Surprising Accuracy in Medical Diagnosis, Highlights Key Limitations
- Navigating Regulatory Hurdles for AI Integration in Healthcare: A Global Perspective
The study concludes that AI-generated patient images are not yet suitable for uncritical use in medical contexts. It emphasizes the urgent need for new strategies to counteract these biases and ensure adequate demographic representation in future software models. The authors recommend that scientific and non-scientific publications explicitly state whether patient images are real or AI-generated, urging caution and careful evaluation before their deployment in healthcare settings.
