TLDR: A new study reveals that ordinary people’s assessment of AI fairness is far more complex and nuanced than traditional expert-driven methods. In a credit rating scenario, non-AI experts considered a broader range of features, tailored fairness metrics to specific contexts, set stricter unfairness thresholds, and even preferred custom fairness definitions. The research highlights the importance of incorporating diverse stakeholder perspectives to align technical AI fairness with human-centered values, suggesting that current expert practices may fall short of public expectations.
Assessing fairness in artificial intelligence (AI) systems has traditionally been the domain of AI experts. These specialists typically select specific features to protect, define fairness metrics, and set acceptable thresholds for what is considered fair. However, a recent study delves into a crucial, yet often overlooked, aspect: how ordinary stakeholders—individuals affected by AI outcomes but without AI expertise—perceive and assess fairness.
The research, titled “I think this is fair”: Uncovering the Complexities of Stakeholder Decision-Making in AI Fairness Assessment, was conducted by Lin Luo, Yuri Nakao, Mathieu Chollet, Hiroya Inakoshi, and Simone Stumpf. Their work highlights that when non-experts are empowered to make decisions about AI fairness, their approaches are far more nuanced and complex than the standard expert-driven methods.
Understanding Stakeholder Perspectives on AI Fairness
The study involved 30 participants, none of whom had formal AI training or professional experience. They were placed in a simulated credit rating scenario and tasked with deciding which features should be prioritized for fairness, which metrics to use, and what levels of unfairness were acceptable. The findings revealed several key differences from conventional expert practices:
- Stakeholders considered a much broader range of features for fairness assessment, extending beyond just legally protected characteristics like age or gender.
- They preferred to tailor fairness metrics to specific contexts and features, rather than applying a single, ‘one-size-fits-all’ metric.
- Their fairness thresholds were often stricter than those typically set by AI experts or legal standards.
- Many participants expressed a desire to design their own customized fairness definitions.
Key Findings: A Deeper Dive into Stakeholder Decisions
The research identified six recurring patterns in how stakeholders approached AI fairness:
1. Broader and Dynamic Feature Choices: Participants identified features as fairness-critical based on their contextual relevance, not just their legal protection status. While protected features like ‘Gender’ and ‘Age’ were important, non-protected features such as ‘Telephone’ or ‘Purpose’ were also frequently prioritized, indicating a holistic view of potential bias.
2. Preference for Individual-by-Individual Fairness: Contrary to the common focus on group-level metrics in AI governance, stakeholders often gravitated towards individual-centered metrics like Counterfactual Fairness. This suggests a strong desire for AI systems to treat each person fairly based on their unique circumstances.
3. Tailoring Metrics to Features: A significant observation was that participants often chose different fairness metrics for different features. For example, Counterfactual Fairness was frequently chosen for ‘Age’ and ‘Telephone’, while Equalized Odds was popular for ‘Foreign Worker’. This contrasts with the expert tendency to apply the same metric across various features.
4. Favoring Custom Metrics: When predefined metrics didn’t align with their perceptions, participants actively sought to define their own. This often involved combining existing metrics to achieve a more comprehensive assessment, reflecting a need for flexible, context-specific fairness definitions.
5. Setting Stricter Thresholds: Stakeholders generally demanded a higher level of fairness, setting notably stringent thresholds for acceptable unfairness. While they allowed for some tolerance, their limits were often well below the 10-20% difference commonly accepted in technical or legal standards.
6. Shifts in Fairness Priority: Participants’ fairness judgments were not static. As they engaged more deeply with the assessment process and understood potential trade-offs, their priorities for features and metrics sometimes shifted, highlighting the dynamic nature of human fairness perception.
Also Read:
- Steering Data for Fairer AI: A New Approach to Bias Reduction
- Decoding Slang: How AI’s Informal Language Differs from Human Expression
Implications for AI Development
These findings underscore the critical need for a human-centered approach to AI fairness. Current expert-led practices, while technically sound, may not align with societal values and public expectations. Incorporating stakeholder input can lead to more legitimate, transparent, and trustworthy AI systems. The study also offers actionable design implications for future interactive tools that can empower non-experts to participate meaningfully in AI fairness assessments.
The researchers developed a prototype system to facilitate this participatory process, allowing stakeholders to explore features, metrics, and thresholds in a credit rating scenario. Despite the inherent complexity of the tasks, participants generally found the system straightforward and confidence-enhancing, suggesting that with appropriate design, non-experts can effectively engage in these critical evaluations. For more details, you can read the full research paper here.
