TLDR: A study introduces the Moral Dilemma Dataset to compare human and LLM moral judgments. It finds LLMs align with humans only under high consensus and rely on a narrower set of moral values, creating a “pluralistic moral gap.” A new method, Dynamic Moral Profiling (DMP), significantly improves LLM alignment and value diversity by steering models with human-derived value profiles.
Large Language Models (LLMs) are increasingly becoming a source of advice for people navigating complex moral decisions. However, a crucial question arises: how well do these AI models truly align with the diverse and often nuanced moral judgments of humans?
A recent research paper titled “The Pluralistic Moral Gap: Understanding Judgment and Value Differences between Humans and Large Language Models” by Giuseppe Russo, Debora Nozza, Paul Röttger, and Dirk Hovy delves into this very question. The study highlights a significant “pluralistic moral gap” between how humans and LLMs approach moral dilemmas, revealing differences in both the distribution of judgments and the diversity of values expressed.
Understanding the Moral Dilemma Dataset
To investigate this gap, the researchers introduced the Moral Dilemma Dataset (MDD). This unique benchmark comprises 1,618 real-world moral dilemmas, meticulously gathered from an online advice forum. Unlike previous studies that often relied on simplified or artificial scenarios, the MDD uses dilemmas that reflect the kind of complex, everyday moral questions people genuinely seek guidance on. Each dilemma in the dataset is paired with a range of human moral judgments, including a simple “acceptable” or “unacceptable” evaluation, along with a free-text explanation of the reasoning behind it.
How LLMs Compare to Human Judgments
The study first examined how LLMs align with human moral judgments. They found that LLMs perform reasonably well when there’s a high level of agreement among humans on a particular dilemma. In situations where most people agree on what’s right or wrong, the models tend to reproduce those majority opinions. However, this alignment sharply declines when human disagreement increases. In ambiguous scenarios, where humans offer a wide spectrum of views, LLMs often default to a single, dominant judgment, failing to capture the rich diversity of human perspectives.
The Diversity of Moral Values
Beyond just the binary judgment, the researchers looked at the underlying moral values that humans and LLMs express when justifying their decisions. Using a comprehensive taxonomy of 60 moral values, they analyzed thousands of value expressions from both human and model rationales. The findings were striking: while LLMs often evoke similar top values as humans (like autonomy, care, and respect), they rely on a much narrower set. The top 10 values account for over 80% of all value mentions in LLM responses, compared to only about 35% in human responses. This indicates that humans draw upon a significantly broader range of moral considerations.
Values such as inclusivity, communication, and child welfare were found to be disproportionately underrepresented in LLM judgments compared to human ones. This systematic difference in the diversity of values is what the researchers term the “pluralistic moral gap.”
Also Read:
- Building Ethical AI: A New Approach to Moral Decision-Making in Autonomous Agents
- Unpacking AI’s Social Smarts: Why Small Language Models Struggle with Generalizable Theory of Mind
Bridging the Gap with Dynamic Moral Profiling
To address this identified gap, the paper proposes an innovative prompting method called Dynamic Moral Profiling (DMP). This technique works by guiding LLMs to reason using “moral profiles” – topic-sensitive distributions of values derived directly from human rationales. Essentially, DMP helps the models consider a wider, more human-like array of values relevant to a given dilemma.
The results for DMP were promising. It significantly improved the alignment between LLM and human moral judgments, reducing the average difference by 64.3% compared to the best previous methods. This improvement was particularly noticeable in those challenging, low-consensus dilemmas where LLMs previously struggled. Furthermore, DMP successfully enhanced the diversity of values expressed by the models, making their reasoning more pluralistic and closer to human thought processes.
While LLMs show promise in providing moral guidance, this research underscores that they still have a tendency to lean towards majority opinions and overlook the rich diversity of human moral reasoning, especially in complex situations. Tools like Dynamic Moral Profiling offer a crucial step towards developing AI systems that can offer more nuanced, pluralistic, and truly human-aligned moral advice. You can read the full research paper for more details here.
