TLDR: A new method for sequential recommender systems tackles the “cold start” problem for new items by adding a small, trainable adjustment (a “delta”) to frozen content-based embeddings. This allows item representations to adapt to user interactions without losing their original content-based meaning, leading to better recommendations for both new and existing items across various data types like text and audio.
Recommender systems have become an indispensable part of our daily digital lives, guiding us through vast catalogs of products, movies, music, and more. These systems learn from our past interactions to predict what we might like next. However, they face a significant challenge known as the “cold start problem,” especially when new items are introduced. Imagine a brand-new song or product; without any prior user interactions, the system struggles to recommend it effectively because it lacks the data to learn its preferences.
Traditional approaches to this problem often rely on content-based features, such as textual descriptions for products or audio characteristics for music. The idea is to use these features to create initial representations, or “embeddings,” for new items. While this helps, it introduces a dilemma: if these content-based embeddings are kept rigid (frozen), the model can’t fully adapt them to how users actually interact with items. On the other hand, if they are allowed to change too much during training (fine-tuning), they might drift so far from their original content meaning that they no longer accurately represent the new item, hurting recommendations for those very cold-start items.
Researchers Anton Pembek, Artem Fatkulin, Anton Klenitskiy, and Alexey Vasilev from Sber AI Lab and universities in Moscow have proposed a novel solution to this limitation. Their paper, titled “Let It Go? Not Quite: Addressing Item Cold Start in Sequential Recommendations with Content-Based Initialization,” introduces an innovative method that allows item representations to adapt without losing their foundational content-based structure. You can read the full paper here.
The core of their approach involves two components for each item’s embedding. First, a “frozen content embedding” is derived from the item’s metadata (like text or audio features) and remains fixed. Second, a small, “trainable delta” vector is added to this frozen embedding. This delta vector is what the model learns and adjusts during training, but its size is carefully constrained. This setup ensures that the final item representation can be fine-tuned to reflect user interaction patterns, yet it always stays close to its original content-based meaning. Think of it as giving the model just enough flexibility to adapt without letting the item’s identity drift away.
This method offers several key advantages. It significantly improves the quality of recommendations for cold-start items, which are items with few or no prior interactions. Crucially, it achieves this without negatively impacting the performance for “warm” items—those that have been seen and interacted with frequently. The researchers demonstrated the effectiveness of their approach across various datasets and modalities, including e-commerce datasets with textual descriptions and a music dataset with audio-based representations, proving its versatility.
Experiments showed consistent improvements in metrics like Hit Rate (HR@10) and Normalized Discounted Cumulative Gain (NDCG@10) for cold items. For instance, on the Amazon-M2 dataset, their method showed a notable increase in HR@10 for cold items compared to baselines. They also found an optimal range for the maximum norm of the trainable delta, indicating a sweet spot where the model has enough flexibility to learn without distorting the original content representation. The approach also proved beneficial when cold items were present within a user’s input sequence and for low-frequency items, further highlighting its robustness.
Also Read:
- New Framework Boosts Temporal Knowledge Graph Predictions for Evolving Data
- Humanoid Robots Master Locomotion with Limited Sensory Data
While the proposed method delivers superior recommendation quality, the authors acknowledge that maintaining a second embedding vector for each item introduces some additional training cost and memory overhead. Future work could explore ways to optimize this, perhaps by reducing embedding sizes or extending the study to even more diverse recommendation scenarios. Nevertheless, this research presents a promising step forward in making recommender systems more robust and effective in handling the constant influx of new content.
