TLDR: The CUPID benchmark evaluates how well Large Language Models (LLMs) can infer and apply dynamic, context-dependent user preferences from interaction histories. It reveals that current state-of-the-art LLMs struggle to accurately infer preferences from multi-turn feedback and identify relevant past contexts, with performance under 50% precision and 65% recall. The study suggests that improving LLMs’ ability to retrieve relevant context and reason about implicit preferences from interactions is crucial for better personalization.
Large Language Models, or LLMs, are becoming increasingly integrated into our daily lives, assisting users with a wide range of tasks and conversations. A key challenge for these models is to align with human values and preferences. Traditionally, personalization of LLMs has assumed that users have fixed preferences that apply across all tasks. However, in reality, human preferences are dynamic and change depending on the specific situation or context.
Imagine a researcher who needs help refining a paper. Their preference for how the LLM should assist might differ based on who they are collaborating with. With one colleague, they might prefer arguments based on classical methods, while with another, they might embrace computational approaches. This highlights the need for LLMs to understand these shifting, context-dependent preferences.
To address this, a new benchmark called CUPID (Contextual User Preference Inference Dataset) has been introduced. CUPID is designed to evaluate how well LLMs can infer a user’s preferences that are tied to different contexts, based on their past interactions. It comprises 756 human-curated interaction session histories between users and LLM-based chat assistants. In each session, a user makes a request within a specific context and provides feedback over multiple turns, gradually revealing their preferences.
The benchmark assesses two main capabilities of LLMs: first, their ability to infer the correct contextual preference given a new request and prior interactions; and second, their ability to generate a response that satisfies this inferred preference. The data for CUPID is generated through a sophisticated pipeline that creates diverse and rich interaction sessions. This involves generating unique user personas, defining context factors (like people, objects, or locations that influence expectations), and creating contextual preferences. Multi-turn dialogues are then simulated where a user persona evaluates an AI assistant’s responses and provides feedback, indirectly revealing their preferences.
Experiments conducted with 10 different state-of-the-art LLMs, including both open-source and proprietary models, revealed that current LLMs struggle significantly with inferring user preferences from multi-turn interactions. Models achieved under 50% precision and 65% recall in identifying the correct preferences. A key finding was that LLMs often fail to recognize which previous context is relevant to a new request. However, performance generally improved with larger model sizes and stronger reasoning capabilities.
Interestingly, the study found that models performed better in ‘Changing’ instances, where a user’s preference for a specific context factor evolved over time, compared to ‘Contrastive’ instances, where conflicting preferences existed across different contexts. This suggests that models tend to prioritize the most recent interactions when inferring preferences. Error analysis showed that models frequently made mistakes by inferring preferences from irrelevant contexts, performing only shallow inferences based on the current request, or even hallucinating preferences.
The research also explored the impact of providing summaries of interaction sessions to the LLMs. While summaries offered minimal gains for already strong models (and sometimes even slightly lowered their performance), they substantially boosted weaker models, bringing them closer to the performance of more advanced ones. This indicates that summaries can help smaller models process and extract preferences from each session, but might lead to some information loss for larger, more capable models.
Also Read:
- Navigating User Experience in Conversational Recommender Systems: A Systematic Review
- Assessing AI’s Real-World Travel Planning Skills: Introducing TripTailor
The findings from CUPID highlight several practical implications for developing more personalized LLMs. It suggests the importance of integrating retrieval techniques that can identify and focus on prior interactions relevant to the current context. For smaller or local LLMs, caching summaries of interaction sessions, focusing on context and preferences, could be beneficial. Finally, it emphasizes the need to train or fine-tune models to perform deeper reasoning about users’ underlying preferences during multi-turn interactions, rather than just relying on surface-level expressions. This work, detailed further at arxiv.org/pdf/2508.01674, provides a valuable resource for advancing conversational AI systems towards truly interactive and contextual personalization.
