TLDR: A new research paper introduces CONFLICTSCOPE, an automated system to create and evaluate scenarios where large language models (LLMs) face conflicts between different values. The study found that LLMs prioritize values differently in open-ended conversations compared to multiple-choice questions, often shifting from protective values like harmlessness to personal values like user autonomy. It also demonstrated that system prompts can moderately improve LLM alignment with desired value rankings.
Large Language Models (LLMs) are becoming integral to our daily lives, assisting with a vast array of tasks. As their influence grows, understanding the values that guide their actions is crucial. However, these powerful AI assistants often encounter situations where different desirable values come into conflict, forcing them to make difficult trade-offs. Traditional methods for evaluating LLM alignment often fall short in capturing these complex moral dilemmas.
A new research paper, “GENERATIVE VALUE CONFLICTS REVEAL LLM PRIORITIES,” introduces an innovative solution called CONFLICTSCOPE. Developed by researchers Andy Liu, Kshitish Ghate, Mona Diab, Daniel Fried, Atoosa Kasirzadeh, and Max Kleiman-Weiner from Carnegie Mellon University and the University of Washington, this automated pipeline aims to shed light on how LLMs prioritize different values when faced with a conflict. You can read the full paper here.
What is CONFLICTSCOPE?
CONFLICTSCOPE is designed to automatically generate realistic scenarios where an LLM assistant must choose between two conflicting values from a user-defined set. Unlike previous datasets that rarely feature genuine value conflicts, CONFLICTSCOPE specifically crafts these challenging situations. It then evaluates the LLM’s free-text responses to these scenarios to determine a ranking of its inherent value priorities.
How Does It Work?
The pipeline operates in several key stages:
1. Scenario Generation: Instead of labeling existing dilemmas, CONFLICTSCOPE takes a “top-down” approach. Given two values (e.g., helpfulness and harmlessness), it uses a powerful LLM (Claude 3.5 Sonnet) to create high-level summaries of potential conflict scenarios. These summaries include the user context, an action the LLM could take, and the benefits and harms under each conflicting value. This process uses various prompt templates to ensure a diverse range of mild to strong conflicts.
2. Scenario Elaboration and Deduplication: After generating summaries, the system removes duplicates to ensure unique scenarios. Then, the same LLM elaborates on each summary, creating detailed descriptions, user personas, and two distinct actions—one supporting each value.
3. Scenario Filtering: To ensure quality and realism, CONFLICTSCOPE employs an LLM-as-a-judge (GPT-4.1) to filter scenarios based on six criteria: realism, specificity, action feasibility, scenario impossibility (ensuring both actions can’t be taken simultaneously), action value-guidedness (confirming actions align with intended values), and genuine dilemma (ensuring no obvious “right” choice). This filtering process is human-validated, ensuring the generated scenarios are truly challenging and relevant.
4. Open-Ended Evaluation: This is a crucial departure from traditional multiple-choice evaluations. CONFLICTSCOPE simulates a user (using GPT-4.1) who interacts with the target LLM in an open-ended conversation based on the scenario. Another judge LLM then analyzes the target model’s free-text response to determine which of the two conflicting actions it most closely resembled. This method aims to reveal more stable and realistic value preferences, as multiple-choice evaluations can be sensitive to minor setup differences.
Key Findings: What CONFLICTSCOPE Revealed
The research yielded several significant insights into LLM behavior:
More Challenging Scenarios: CONFLICTSCOPE-generated datasets were found to be more effective at eliciting disagreement among different LLMs compared to existing moral decision-making and alignment datasets. This indicates that the pipeline successfully creates scenarios that force models to make difficult trade-offs, rather than choosing between indifferent options.
Shift in Value Priorities: A striking finding was the difference between “expressed preferences” (in multiple-choice settings) and “revealed preferences” (in open-ended interactions). In multiple-choice evaluations, models often prioritized protective values like harmlessness. However, in open-ended, realistic conversations, they shifted significantly towards supporting personal values such as user autonomy or helpfulness. For instance, in helpfulness-harmlessness conflicts, most models ranked helpfulness over harmlessness in open-ended settings, a reversal from their multiple-choice choices.
Steerability with System Prompts: The study also explored whether LLM behavior could be steered towards a target value ranking using system prompts. By including detailed value orderings and conflict resolution guidelines in the system prompt, models showed an average of 14% improvement in alignment with the desired ranking. This demonstrates that while there’s room for improvement, system prompting can meaningfully alter how LLMs prioritize values under conflict.
Also Read:
- Unpacking AI’s Social Compass: How Language Models Navigate Role Conflicts and Reveal Hidden Biases
- A Cost-Effective Way to Measure LLM Intelligence: The Consistency Score
Why This Matters
The CONFLICTSCOPE pipeline offers a robust and automated way to evaluate the complex ethical decision-making of LLMs. By moving beyond static benchmarks to more realistic, open-ended interactions, researchers can gain a more accurate understanding of how these models will behave in real-world deployments. This work provides a vital foundation for future research in AI alignment, enabling developers to design LLMs that better reflect desired human values and ethical priorities.
