TLDR: LaMAR is a framework that uses Large Language Models (LLMs) to automatically generate rich semantic context for user-item interaction sequences in sequential recommender systems. By inferring latent aspects like usage scenarios or thematic summaries from existing metadata, LaMAR augments sparse interaction data. This LLM-generated information consistently improves the performance of recommendation models across various datasets and can also enhance LLM-based recommenders, demonstrating LLMs’ utility as intelligent data augmenters.
In the evolving landscape of recommendation systems, understanding user preferences from their past interactions is crucial. However, traditional sequential recommenders often struggle with sparse data, where item histories lack rich contextual information. This limitation makes it difficult for models to infer nuanced user interests, especially for new users or less popular items. A new research paper introduces LaMAR, a framework designed to overcome this challenge by leveraging the power of Large Language Models (LLMs) to semantically enrich user-item interaction sequences.
The paper, titled “Language Models as Semantic Augmenters for Sequential Recommenders,” by Mahsa Valizadeh, Xiangjue Dong, Rui Tuo, and James Caverlee from Texas A&M University, proposes a novel data-centric approach. Instead of relying solely on existing, often minimal, item metadata, LaMAR uses LLMs to generate auxiliary contextual signals. These signals infer latent semantic aspects of user intent and item relationships, such as usage scenarios, item intents, or thematic summaries, thereby adding greater contextual depth to the original interaction sequences.
How LaMAR Works: Two Key Stages
LaMAR operates in two primary stages to enhance sequential recommendation systems:
The first stage focuses on **Semantic Signal Generation via LLMs**. Here, LLMs are prompted to infer semantic signals that are not explicitly present in the dataset but can be derived from item metadata like titles, brands, and categories. For example, for a product in the “Pet Supplies” category, an LLM might infer a “Pet’s Specific Need” signal, detailing the intended use of the product. Similarly, for “Video Games,” it could generate a “Gameplay Experience Focus.” This process uses a few-shot prompting strategy, guiding the LLM to produce contextually informative natural language phrases or sentences that capture latent aspects of an item.
The second stage involves **Signal Integration and Model Alignment**. The LLM-generated semantic signals are incorporated into the input representation of sequential recommendation models. This means that each item in a user’s interaction history is augmented with its new semantic signal, creating an enriched sequence. This allows the recommendation model to consider both explicit metadata and the LLM-inferred semantics when learning user patterns. Furthermore, the sequential recommenders or the LLMs themselves can be fine-tuned using these augmented sequences, aligning their behavior more closely with recommendation objectives and improving their understanding of useful semantics.
Also Read:
- Improving Entity Linking with Contextual Augmentation and Large Language Models
- Optimizing Information Retrieval for AI: A Bandit Approach to Complex Queries
Demonstrated Effectiveness and Generalizability
The researchers conducted extensive experiments across multiple public sequential recommendation datasets, primarily from Amazon product categories like Industrial and Scientific, Musical Instruments, and Video Games. They compared LaMAR against state-of-the-art sequential recommendation baselines, using LLMs such as GPT-4o-mini and Gemini-1.5-flash for signal generation.
The results consistently showed performance gains across key ranking metrics like NDCG@10 and Recall@10. For instance, in the Video Games dataset, LaMAR improved NDCG@10 by over 5% and Recall@10 by over 6% with GPT-generated signals. This indicates that LaMAR helps models rank more relevant items higher and retrieve more relevant items overall. The framework also proved effective in improving LLM-based recommendation models when fine-tuned with the augmented data, though the benefits were less pronounced when generated features were highly similar, suggesting the importance of diverse signals.
An analysis of signal diversity revealed that while some categories might lead to semantically similar phrasing at lower thresholds (e.g., Pet Supplies, Musical Instruments), most generated features were semantically distinct from a large portion of the dataset, demonstrating strong content diversity. This highlights the LLMs’ ability to generate varied and original outputs in less templated categories.
Beyond e-commerce, LaMAR’s generalizability was tested on the MovieLens-1M dataset. By enriching movie items with storyline information and a new semantic signal, “how the movie ends,” generated by GPT-4o-mini, the framework again showed consistent improvements across various metrics. This demonstrates LaMAR’s potential to enhance recommendations in diverse domains.
In conclusion, LaMAR represents a significant step forward in sequential recommendation, offering a scalable and data-centric paradigm where LLMs act as intelligent context generators. By automatically enriching interaction sequences with nuanced semantic information, LaMAR improves the representational capacity of downstream models and contributes a new method for the semi-automatic creation of training data and language resources. You can read the full research paper here: Language Models as Semantic Augmenters for Sequential Recommenders.
