TLDR: A new research paper explores using Large Language Models (LLMs) to create distinct short-term and long-term user profiles from interaction histories for improved recommendation systems. It finds that this approach significantly enhances recommendation quality in domains with active user engagement (like Movies&TV) where preferences are more dynamic, but shows less pronounced benefits in sparser domains (like Video Games) with more stable user profiles. A key benefit is the inherent interpretability provided by natural language profiles and attention weights, offering a clearer understanding of recommendation rationale.
Recommender systems are everywhere, from helping us pick our next movie to suggesting products we might like. But these systems face a big challenge: understanding that our preferences aren’t static. What we like today might be a fleeting interest, while other preferences are deeply ingrained and long-lasting. Traditional methods often struggle to tell the difference, mixing up these short-term desires with stable long-term tastes, which can lead to less accurate and less transparent recommendations.
A new research paper explores how Large Language Models (LLMs) can tackle this problem by creating more nuanced and dynamic user profiles. The core idea is to use LLMs to generate separate textual summaries of a user’s interaction history: one focusing on recent activities (short-term interests) and another on consistent behaviors over time (long-term preferences).
How it Works: LLMs for Temporal User Profiling
The framework involves three main steps. First, an LLM is prompted twice for each user’s interaction history. One prompt is designed to extract recent interests, creating a ‘short-term profile’ in natural language. The other prompt focuses on persistent behaviors, generating a ‘long-term profile’. These textual summaries are then converted into numerical representations, or embeddings, using a model like BERT. Finally, a clever attention mechanism adaptively combines these short-term and long-term embeddings to form a unified user representation. This mechanism also provides ‘attention weights’ that explicitly show whether a recommendation is driven more by recent interests or enduring preferences, offering a pathway to interpretability.
Performance Across Different Domains
The researchers tested this LLM-driven approach on two distinct Amazon domains: Movies&TV and Video Games. The results showed varied effectiveness depending on the domain’s characteristics.
In the Movies&TV domain, which typically has higher user activity and more varied behaviors, the LLM-driven temporal profiling demonstrated significant improvements. It outperformed traditional baselines, showing gains of 17% in Recall@10 and 14% in NDCG@10. This suggests that where user preferences are dynamic and frequently changing, the ability of LLMs to capture these semantic nuances is particularly valuable.
However, in the Video Games domain, characterized by sparser interactions and potentially more stable user profiles, the benefits were less pronounced. While the approach still showed some improvements, another temporal fusion method without LLM-generated semantic profiles sometimes performed similarly or slightly better at smaller K-values. This indicates a trade-off: the detailed temporal modeling offered by LLMs is most impactful when short-term and long-term preferences are clearly distinguishable, which might not always be the case in domains with less frequent or more consistent user engagement.
Also Read:
- Enhancing AI Persona Consistency in Dialogue Simulations
- Understanding Knowledge Dynamics in LLM Explanations with a New Framework
A Step Towards Transparent Recommendations
Beyond just improving prediction accuracy, a significant advantage of this LLM-driven method is its inherent potential for interpretability. The natural language summaries generated by the LLMs provide human-readable profiles of a user’s interests. Coupled with the attention weights that reveal the influence of short-term versus long-term preferences, the system offers a clearer understanding of why a particular recommendation was made. While fully user-facing explanations are a future goal, this work lays a strong foundation for more transparent and trustworthy recommender systems.
An ablation study further confirmed the importance of distinguishing between short-term and long-term preferences, as well as the need for non-linear interaction modeling. Variants that did not separate temporal aspects or used simpler scoring mechanisms showed significant performance drops.
In conclusion, this research highlights the practical capability and inherent interpretability of leveraging LLMs for temporal user profiling. It suggests that while LLMs can significantly enhance recommendation quality, especially in active domains, their application should consider the specific context and user behavior patterns. This work opens new avenues for developing adaptive and transparent recommender systems. You can read the full research paper here: Effectiveness of LLMs in Temporal User Profiling for Recommendation.
