Disability Across Cultures: A Human-Centered Audit of Ableism in Western and Indic LLMs
This work addresses the problem of AI bias in detecting ableism across cultures for people with disabilities, highlighting the need for localized, inclusive standards, though it is incremental in extending existing audit methods to new cultural contexts.
The study investigated whether Western and Indic large language models (LLMs) can adequately recognize ableist harm in non-Western contexts like India, finding that Western LLMs consistently overestimated ableist harm while Indic LLMs underestimated it, and all models were more tolerant of ableism in Hindi and Western framings.
People with disabilities (PwD) experience disproportionately high levels of discrimination and hate online, particularly in India, where entrenched stigma and limited resources intensify these challenges. Large language models (LLMs) are increasingly used to identify and mitigate online hate, yet most research on online ableism focuses on Western audiences with Western AI models. Are these models adequately equipped to recognize ableist harm in non-Western places like India? Do localized, Indic language models perform better? To investigate, we adopted and translated a publicly available ableist speech dataset to Hindi, and prompted eight LLMs--four developed in the U.S. (GPT-4, Gemini, Claude, Llama) and four in India (Krutrim, Nanda, Gajendra, Airavata)--to score and explain ableism. In parallel, we recruited 175 PwD from both the U.S. and India to perform the same task, revealing stark differences between groups. Western LLMs consistently overestimated ableist harm, while Indic LLMs underestimated it. Even more concerning, all LLMs were more tolerant of ableism when it was expressed in Hindi and asserted Western framings of ableist harm. In contrast, Indian PwD interpreted harm through intention, relationality, and resilience--emphasizing a desire to inform and educate perpetrators. This work provides groundwork for global, inclusive standards of ableism, demonstrating the need to center local disability experiences in the design and evaluation of AI systems.