TLDR: A study investigated whether Large Language Models (LLMs) reflect “WEIRD” (Western, Educated, Industrialized, Rich, Democratic) values and how this relates to human rights. Researchers found that while some LLMs (like GPT-3.5 and GPT-4) align more with WEIRD values, others (like BLOOM and Qwen) are less WEIRD. Surprisingly, models with lower WEIRD alignment were 2% to 4% more likely to generate responses violating human rights, particularly concerning gender equality. The research highlights a complex trade-off: increasing cultural diversity in LLMs might inadvertently increase the risk of reproducing discriminatory beliefs, suggesting a need for approaches like Constitutional AI to embed human rights principles.
Large Language Models (LLMs) are becoming increasingly common in generating human-like text, but they often carry societal biases from their training data. A significant concern is the over-representation of what researchers call “WEIRD” populations: Western, Educated, Industrialized, Rich, and Democratic. This bias risks amplifying Western-centric views and marginalizing perspectives from other parts of the world.
A recent study explored this very issue, investigating how closely five widely used LLMs—GPT-3.5, GPT-4, Llama-3, BLOOM, and Qwen—align with WEIRD values and whether their responses conflict with human rights principles. To understand global diversity, the researchers compared LLM responses against the Universal Declaration of Human Rights and three regional charters from Asia, the Middle East, and Africa. You can read the full research paper here: Should LLMs be WEIRD? Exploring WEIRDness and Human Rights in Large Language Models.
The study found that models like GPT-3.5 and GPT-4 showed the highest alignment with WEIRD populations, especially in the Western and Democratic aspects. Llama-3 also showed significant alignment in the Educated dimension. In contrast, BLOOM and Qwen consistently had the lowest alignment across most dimensions, particularly in Western and Industrialized categories. This suggests that BLOOM, with its extensive multilingual training, is comparatively the least WEIRD LLM, offering more balanced responses across different cultural contexts.
To understand why LLMs align with WEIRD values, the researchers analyzed questions where LLM responses were most similar to those from WEIRD countries. Five key themes emerged:
Social and Moral Values
LLMs often reflected ethical stances and social behaviors common in WEIRD countries. For instance, when asked about a man beating his wife, LLMs consistently chose “never justified,” aligning with views in countries like New Zealand, Germany, and the United States.
Political and Governance Attitudes
LLMs showed confidence in governmental organizations like the police, mirroring sentiments in many WEIRD nations such as Germany, Canada, and Singapore.
Social Participation and Trust
The models expressed high confidence in societal organizations like the World Health Organization (WHO), similar to responses from countries like Canada and the Netherlands.
Attitudes Toward Immigration and Diversity
LLMs tended to agree that immigration strengthens cultural diversity, aligning with views in countries like the Netherlands, Germany, and Canada.
Also Read:
- AI Models Outperform Individual Humans in Predicting Everyday Social Norms
- Measuring AI Smarts: IQ, EQ, and Professional Skills for Language Models
Perceptions of Security and Social Order
When asked about police interference in private lives, LLMs responded with “not at all,” reflecting perceptions in countries like Singapore and Germany.
Interestingly, the study revealed a complex trade-off. While reducing WEIRD bias might seem beneficial for cultural representation, models with lower alignment to WEIRD values, such as BLOOM and Qwen, were 2% to 4% more likely to generate outputs that violated human rights. These violations were particularly evident regarding gender and equality. For example, some models agreed with statements like “a man who cannot father children is not a real man” or “a husband should always know where his wife is,” reflecting harmful discriminatory norms.
GPT-4, despite showing a higher degree of WEIRD characteristics compared to Llama-3 and BLOOM, actually exhibited the fewest human rights violations overall. This suggests that being aligned with certain WEIRD values, which often emphasize ethical standards, democratic principles, and inclusive values, may not always be detrimental and can even align with human rights principles.
The findings highlight that simply diversifying LLMs to include more non-Western perspectives doesn’t automatically ensure ethical or inclusive outcomes. Instead, it can introduce new challenges where values common in some non-WEIRD regions may conflict with internationally recognized human rights. This calls for a more nuanced approach to fairness in AI, one that balances equitable representation with the safeguarding of fundamental rights, especially for marginalized groups.
The researchers suggest that approaches like Constitutional AI, which explicitly embed human rights principles into a model’s architecture during training or fine-tuning, could help resolve this tension. By integrating public feedback and constitutional principles, LLMs can be designed to respect global values while minimizing bias and discrimination.
