TLDR: The paper introduces DIVE, the first multimodal dataset for pluralistic alignment in text-to-image (T2I) models. It uses a diverse pool of human raters across 30 demographic intersections to gather nuanced feedback on T2I safety. The research empirically confirms that demographics significantly influence harm perception, revealing differences from conventional evaluations, especially for bias. DIVE provides foundational tools for building more equitable T2I systems by informing data collection, LLM judgment, and model steerability towards diverse human values.
Text-to-image (T2I) models have opened up incredible creative possibilities, allowing us to generate images from simple text prompts. However, their widespread use also brings a significant challenge: the risk of perpetuating harmful biases, stereotypes, violence, and explicit content. The core issue is that what one person considers harmful, another might not, as harm is deeply subjective and tied to individual experiences.
To address this, a new research paper introduces the concept of “pluralistic alignment.” This approach recognizes that human values are diverse and often conflicting, and for AI to be truly aligned, it needs to understand and be steerable towards these multifaceted viewpoints.
Introducing the DIVE Dataset
The researchers have made a significant contribution by introducing a novel dataset called Diverse Intersectional Visual Evaluation (DIVE). This is the first multimodal dataset specifically designed for pluralistic alignment in the T2I domain. DIVE aims to enable a deeper understanding of diverse safety perspectives by gathering extensive feedback from a large pool of human raters.
What makes DIVE unique is its carefully constructed rater pool. It includes 637 human raters across 30 distinct intersectional groups, created by combining different ethnicities (White, Black, Latinx, SouthAsian, EastAsian), age groups (GenZ, Millennial, GenX), and genders (Men, Women). This ensures a balanced representation of diverse backgrounds.
The annotation process in DIVE captures nuanced safety perceptions. Raters use a five-point Likert scale to indicate harmfulness, distinguish between harm perceived personally versus harm perceived by others, and provide open-ended explanations for their ratings. With 20-30 diverse raters assessing each prompt-image pair, the dataset provides reliable, quantitative insights that go beyond the limitations of single-rater or policy-driven evaluations. The DIVE dataset is a large-scale resource, containing 1000 adversarial prompts with over 35,000 harm evaluations, offering fine-grained categorization and rich feedback.
Demographics as a Key Indicator of Viewpoints
A crucial finding from this research is the empirical confirmation that demographic attributes serve as a vital proxy for understanding different viewpoints in the context of T2I safety. The analysis reveals significant, context-dependent differences in how various trisectional demographic groups perceive harm. These perceptions often diverge considerably from conventional, policy-driven safety evaluations.
For instance, the study found that Women raters are more likely to assign higher harm scores than Men raters. Among ethnic groups, Black raters were more likely to give higher harm scores than White raters. When looking at deeper demographic intersections, groups like GenZ-Black and Millennial-Black raters showed higher group cohesion in their safety perceptions compared to broader demographic groupings.
The research also explored how human perspectives vary with different types of content. Simulations showed that Women raters consistently flagged more instances of harm across all violation types (Explicit, Violent, Bias) compared to Men. Ethnicity-based groupings revealed that SouthAsian, Latinx, and Black raters were more likely to flag ‘Bias’ type issues that other groups might deem safe, highlighting specific blind spots in conventional evaluations.
Implications for Building Aligned AI Models
The findings from the DIVE dataset have significant implications for developing more aligned T2I models. The paper discusses strategies for efficient data collection with diverse rater pools and investigates the capacity of Large Language Models (LLMs) to reflect diverse viewpoints when acting as judges. It also examines the steerability of LLMs towards specific diverse perspectives.
When comparing diverse human ratings with existing safety classifiers (like policy raters or automated tools such as LlavaGuard and ShieldGemma), the study found that current safety evaluations often have blind spots, particularly in identifying issues related to ‘Bias,’ which require more contextual and culture-specific information. Diverse raters were more sensitive to these nuanced harms.
The researchers also experimented with steering LLMs to represent specific demographic subgroups. By prompting a multi-modal Gemma model with demographic information (e.g., “take the perspective of a Woman, GenX, Asian”), they observed an improvement in the LLM’s ability to produce ratings aligned with those specific demographic groups, although the overall correlation remains low, indicating room for future work.
Also Read:
- GIFT: A New Defense for Diffusion Models Against Malicious Fine-Tuning
- When Visual Input Challenges VLM’s Internal Facts
Looking Ahead
The DIVE dataset provides a strong foundation for future research into pluralistic alignment. It offers a unique resource for inclusive safety evaluation, enabling a deeper understanding of how diverse human experiences shape perceptions of harm in AI-generated content. This work is a crucial step towards building more equitable and aligned T2I systems that truly account for the varied perspectives of all users. You can learn more about the dataset and the research at the research paper’s page.
