TLDR: This research paper is a scoping review examining how fairness, bias, and equity are addressed in Automatic Speech Recognition (ASR) for African American English (AAE) speakers. It synthesizes 44 interdisciplinary publications from Human-Computer Interaction (HCI), Machine Learning/Natural Language Processing (ML/NLP), and Sociolinguistics. The review identifies a critical gap in governance-centered approaches that prioritize community agency and linguistic justice. In response, it proposes a ‘governance-centered ASR life-cycle’ framework, advocating for participatory oversight and institutional accountability across all stages of ASR development to create more responsible and equitable speech AI systems.
Automatic Speech Recognition (ASR) systems, which power everything from transcription services to virtual assistants, have become ubiquitous in our daily lives. While promising greater access, these systems often fall short for speakers of African American English (AAE), frequently misrecognizing or misinterpreting their linguistic patterns. This issue isn’t just a technical glitch; it reflects and reinforces broader societal inequities about whose voices are valued by AI. A recent scoping review, titled Toward Responsible ASR for African American English Speakers: A Scoping Review of Bias and Equity in Speech Technology, by Jay L. Cunningham, Adinawa Adjagbodjou, Jeffrey Basoah, Jainaba Jawara, Kowe Kadoma, and Aaleyah Lewis, delves into how fairness, bias, and equity are understood and addressed in ASR and related speech technologies for AAE speakers and other diverse communities.
Understanding the Problem of Bias
The research highlights that ASR systems predominantly trained on standardized American English (SAE) datasets struggle with AAE. This leads to higher word error rates, reduced reliability, and misrecognition of culturally specific language for AAE speakers. These failures can manifest as usability issues, erosion of trust, and miscommunication, with potential serious consequences in critical areas like healthcare or policing. The paper emphasizes that these harms are rooted in long-standing language ideologies that devalue non-standard dialects, perpetuating social hierarchies and racialized perceptions.
An Interdisciplinary Look at Fairness
The review synthesizes findings from 44 peer-reviewed publications across Human-Computer Interaction (HCI), Machine Learning/Natural Language Processing (ML/NLP), and Sociolinguistics. Each field offers a distinct perspective:
-
ML/NLP: This domain primarily focuses on quantifying disparities and mitigating bias through performance evaluation and metrics. Researchers here document racial disparities in ASR word error rates and explore technical interventions like counterfactual fairness. However, these approaches often lack engagement with the social and linguistic contexts of AAE.
-
HCI: HCI studies examine ASR fairness through the lens of user experience, trust, and emotional impact. They reveal how ASR misrecognition can feel like a microaggression, leading to code-switching, self-silencing, and distrust among Black users. While advocating for inclusive design, HCI research often doesn’t directly influence model-level technical decisions.
-
Linguistics and Sociolinguistics: These fields emphasize how ASR systems encode structural linguistic inequalities, arguing that AAE is systematically devalued in datasets. They critique the colonial legacies of linguistic extraction and use theories of linguistic capital to show how ASR reinforces dominant language ideologies.
A key finding across these disciplines is that while ASR systems disproportionately fail marginalized speakers, particularly those using AAE, there’s a significant gap in bridging these technical, interactional, and ideological dimensions. Participatory approaches, which involve community input, are largely absent in ML/NLP, suggesting a need for more integrated research.
Data Practices for Inclusive ASR
The review examines data practices across collection, curation, annotation, and model training. Inclusive data collection involves gathering speech from diverse AAE speakers, sometimes through community-driven efforts. Responsible data curation means creating datasets that document AAE variation with transparent tagging. Annotation is a critical stage where bias can enter; using AAE speakers as annotators and applying race-priming can improve interpretive fairness. For model training, dialect-specific approaches, like retraining models on AAE-specific data, show promise. However, the paper notes that participatory governance, community consent, and ethical alignment remain underdeveloped throughout the data lifecycle.
Practical Recommendations for Equitable Systems
The paper offers several practical recommendations for designing and governing more AAE-inclusive ASR systems:
-
Community Engagement: Involve linguistic communities in defining problems, goals, and design values from the outset.
-
Ethical Data Collection: Implement intentional, consent-based data collection that respects community norms and linguistic realities, potentially through partnerships with historically Black institutions.
-
Responsible Curation and Annotation: Recruit annotators familiar with AAE and apply dialect/race priming to reduce bias, ensuring transparency in training and commitment to dialectal accuracy.
-
Inclusive Model Training and Evaluation: Develop models that accommodate linguistic variation, using techniques like dialect-specific pretraining, and augment traditional metrics with qualitative evaluations of user trust and microaggression impacts.
-
Deployment and Accountability: Design systems with user autonomy in mind, allowing customization and providing mechanisms for feedback and redress. Establish community-based review panels for high-stakes contexts.
-
Reflexive Evaluation: Incorporate continuous reflection, diversity audits, and community accountability protocols, building interdisciplinary teams with equity goals embedded in project governance.
Also Read:
- Bridging the Speech Gap: How ASR Models Perform in Real-World Robot Interactions
- Beyond Bias: Integrating Intersectional Feminist Theory into Algorithmic Fairness
A Governance-Centered ASR Life-Cycle
A central contribution of the review is the proposal of a governance-centered ASR life-cycle. This framework moves beyond narrow technical fixes to embed community agency, participatory oversight, and institutional accountability across all stages of ASR development. It identifies participatory checkpoints from problem definition and data sourcing to model training, evaluation, deployment, and post-deployment governance. This approach treats the fluidity and variation of AAE not as a problem to be minimized, but as a feature to be preserved through continuous updates and community-led monitoring.
The framework calls for a structural shift in how ASR systems are built, ensuring that fairness is co-defined, power is redistributed, and community stewardship guides the future of voice-based technologies. This includes community ownership of datasets, co-designing evaluation criteria, and establishing longitudinal oversight bodies with real decision-making authority. The authors argue that this approach is crucial for countering epistemic exploitation and algorithmic erasure, ensuring ASR systems are adaptable to real-world linguistic change, and ultimately, advancing a justice-oriented vision for speech AI.
