TLDR: This paper argues for a paradigm shift in how Human Label Variation (HLV)—legitimate disagreement in data annotation—is treated in NLP. Historically dismissed as noise, then recognized as a signal for robustness, HLV should now be considered a “Selbstzweck” (an end in itself) for building human-centered AI. The authors propose proactively incorporating HLV into preference datasets by carefully selecting diverse annotator pools, releasing individual annotator-level preferences, and utilizing heterogeneous feedback protocols to preserve the pluralism of human values in AI alignment.
In the rapidly evolving world of Natural Language Processing (NLP), a critical concept known as Human Label Variation (HLV) is gaining significant attention. HLV refers to the legitimate disagreements that arise when different people annotate data, reflecting the genuine diversity of human perspectives rather than simple errors. For a long time, these variations were dismissed as mere “noise” that needed to be eliminated to achieve a single “gold standard” label. However, recent research, including a compelling paper by Shanshan Xu, Santosh T.Y.S.S, and Barbara Plank, argues for a profound shift in how we perceive and utilize HLV.
Historically, NLP systems were built with the assumption that there was always one correct answer. Annotation pipelines were designed to minimize disagreement, often using methods like majority vote aggregation, which effectively flattened diverse human viewpoints into a single, often misleading, consensus. This approach risked erasing the rich pluralism of human values that alignment aims to preserve.
The Evolution of HLV: From Noise to Signal to Selbstzweck
The paper traces the conceptual evolution of HLV through three distinct stages, paralleling the broader integration of human pluralism into NLP research and system design:
- Human Adjacent (Before 1990s): In the early days of NLP, humans were largely peripheral. Systems focused on well-defined tasks like text retrieval or syntactic parsing, relying on expert knowledge and handcrafted rules. Disagreement was seen purely as error, to be eliminated to maintain a “gold standard.” Ordinary people had little input in system design or evaluation.
- Human Aware (1990s Onward): With the rise of the web and deep learning, NLP applications became more embedded in everyday life, tackling subjective problems like fake news or hate speech detection. Large-scale crowdsourcing introduced more HLV. Researchers began to recognize the societal impact of NLP, shifting focus to trustworthiness, interpretability, bias, and fairness. Systems became “human aware,” targeting end-users but still largely designed by organizations.
- Human Centered (Since ChatGPT): The advent of large language models (LLMs) like ChatGPT has transformed NLP into a domain of open-ended, interactive systems. LLMs engage in inherently subjective activities, making human preferences central to their alignment. The paper argues that in this “human centered” era, HLV must be treated as a “Selbstzweck” – a philosophical term meaning “an end in itself.” This means preserving HLV as an embodiment of human pluralism should be an intrinsic design principle for AI systems, rather than just a means to improve model robustness.
Also Read:
- Protecting Human Feedback Privacy in AI Alignment
- Unpacking LLM Judge Capabilities: Human-Like vs. Super-Consistent AI
Integrating HLV into Preference Datasets
The authors emphasize that while much effort has focused on evaluation frameworks and reward-learning algorithms, the foundational preference datasets that encode human judgments remain underexamined. Current datasets often aggregate multiple annotations into a single label, thereby flattening diverse perspectives. To address this, the paper proposes actionable strategies for embedding HLV into future alignment pipelines:
- Carefully Select the Annotator Pool: To truly reflect pluralism, the annotator pool should mirror the diversity of the target population. This involves defining the target population, recruiting with stratified quotas, and using multi-channel strategies to include underrepresented groups. Collecting annotator metadata is also crucial for balanced assignments and post-hoc corrections.
- Release Annotator-Level Preferences: Instead of aggregating labels into a single consensus, datasets should release individual annotator preferences. This allows for more nuanced reward models that capture uncertainty and heterogeneity, leading to safer and more representative AI policies. Aggregating labels can harm pluralistic alignment by rewarding decisive choices even in cases of low agreement.
- Leverage Diverse Feedback Protocols: Current preference datasets often rely on simple relative (pairwise comparisons) or absolute (Likert-scale ratings) feedback. While simple, relative feedback can favor assertive responses and obscure nuanced judgments. The paper advocates for combining diverse feedback types, including ratings, comparisons, demonstrations, corrections, and descriptive evaluations, to more faithfully capture the richness of human preferences and values.
By proactively incorporating HLV into preference datasets, the NLP community can move towards building genuinely human-centered AI systems that are not only aligned but also adaptive to the rich diversity of human needs, contexts, and worldviews. This position paper serves as a crucial call to action, urging researchers to recognize and preserve human pluralism as a core design principle in AI development. You can read the full research paper here.
