TLDR: An interview study with 25 AI practitioners reveals significant challenges in addressing disability inclusion within AI product development. Key issues include AI models perpetuating stereotypes, features being inaccessible, lack of expert knowledge, difficulties involving disabled users and their data, and conflicting review processes that often overlook disability. Despite resource constraints, practitioners employ informal advocacy, volunteer efforts, and strategic leadership buy-in to push for better accessibility, highlighting an urgent need for integrated education and policy changes in responsible AI.
The rapid rise of generative artificial intelligence (AI) is transforming technology, but it also presents significant challenges for disability inclusion. A recent study, titled “Accessibility people, you go work on that thing of yours over there”: Addressing Disability Inclusion in AI Product Organizations, delves into how AI practitioners navigate these complexities within their organizations. The research highlights that decisions made during AI system development can disproportionately affect people with disabilities, often because current accessibility guidelines don’t fully address novel AI interactions and outputs.
The study, conducted through interviews with 25 AI practitioners across various roles—including engineering, research, user experience (UX), and responsible AI—uncovered several key areas of friction. Practitioners reported encountering issues where AI models generated problematic representations of people with disabilities, such as distorted images or perpetuating stereotypes. For instance, an image model might add extra limbs to a disabled person or depict all disabled individuals as “angry old men in wheelchairs,” reflecting biases in the training data.
Beyond representation, many AI-powered features simply don’t work well for disabled users. Voice interfaces, for example, often pose significant barriers for individuals with speech disabilities. Similarly, video chat features designed to automatically frame faces can inadvertently cut off hand movements crucial for sign language communication. Basic accessibility features, like captions for voice products, are sometimes overlooked due to aggressive product launch schedules, leading to obvious regressions in accessibility.
A major obstacle identified was the lack of specialized knowledge and access to experts. Many AI practitioners, especially those not in dedicated accessibility roles, admitted to lacking expertise in how accessibility applies to AI products. Organizational changes, such as team reconfigurations and layoffs, further complicate efforts to find and consult with accessibility experts. This often leaves teams without clear guidance on how to address disability-related concerns.
Involving people with disabilities (PWD) and their data in the development process also presents significant hurdles. Researchers found it difficult to access datasets that adequately represent disabled individuals. Recruiting disabled participants for user testing is often seen as a “niche” activity, requiring additional justification and resources. Furthermore, existing recruitment policies and screening questions may not capture the specific disability categories relevant to AI projects, making it challenging to find the right participants. Lengthy legal and review processes, intended to protect participants, can also delay or prevent crucial research with disabled stakeholders.
The study also revealed conflicts and ambiguities in review processes. Companies often have separate guidelines for traditional accessibility and “responsible AI” (RAI). Issues impacting disabled users might fall into a grey area, with no clear process for resolution. RAI evaluations, while important, frequently prioritize identity categories like race and gender, often overlooking disability. This can lead to situations where accessibility concerns, such as providing full content descriptions to blind users, might clash with RAI policies designed to block “inappropriate” content, creating a dilemma about which policy takes precedence.
The question of responsibility for addressing these issues often becomes a “hot potato,” particularly when multiple teams (e.g., hardware, software, AI model developers) are involved. There’s a perceived lack of upfront budgeting for accessibility, with teams expecting others to bear the cost. This confusion is exacerbated by the modular nature of modern AI development, where foundation models are integrated into products by different teams without clear guidance on downstream accessibility impacts.
Despite these challenges, practitioners shared strategies for making progress with limited resources. Some individuals informally stepped into accessibility advocacy roles, demonstrating the need for formal positions. Volunteer initiatives, such as “red teaming” models with diverse identity groups and company-wide affinity groups for disabled employees, have also been instrumental in advocating for inclusion. Crucially, gaining buy-in from leadership often involves framing accessibility projects as having a “curb cut” effect—innovative solutions designed for accessibility that ultimately benefit a broader user base.
Also Read:
- Unmasking Hidden Biases: How LLMs Infer Demographics from Disability Cues
- Beyond Bias: Integrating Intersectional Feminist Theory into Algorithmic Fairness
The research underscores the urgent need to re-evaluate education and training around responsible AI and accessibility, ensuring that people with disabilities are considered valuable stakeholders throughout the AI development and release process. It calls for expanding our knowledge base on how AI impacts PWD and integrating this understanding into mainstream AI practices. For more details, you can read the full research paper here: Addressing Disability Inclusion in AI Product Organizations.
