TLDR: A new research paper introduces Pctx, a personalized context-aware tokenizer for generative recommendation models. Unlike existing static methods that assign fixed semantic IDs to items, Pctx incorporates a user’s historical interactions to tokenize the same item into different semantic IDs based on individual context. This allows generative recommendation models to capture diverse user interpretations and produce more personalized predictions, demonstrating up to an 11.44% improvement in NDCG@10 over non-personalized baselines.
Generative recommendation (GR) models are changing how we think about personalized suggestions. Instead of just using unique IDs for items, these models break down each user action into a few discrete tokens, called semantic IDs. This approach offers several benefits, including better memory efficiency, improved scalability, and the potential to combine different stages of recommendation, like retrieval and ranking, into a single system.
However, a significant challenge with current GR models is that their tokenization methods are often static and non-personalized. This means that semantic IDs are typically created based solely on an item’s features, like its title or description, assuming that all users perceive item similarities in the same way. In reality, a single item can be interpreted very differently depending on a user’s unique preferences and past interactions. For example, one person might buy a watch as a gift, another as an investment, and a third simply because they like its appearance. Current models struggle to capture these diverse interpretations, leading to less personalized recommendations.
To address this limitation, a new research paper introduces a novel approach called Pctx: Tokenizing Personalized Context for Generative Recommendation. This method proposes a personalized context-aware tokenizer that takes into account a user’s historical interactions when generating semantic IDs. The core idea is to allow the same item to be tokenized into different semantic IDs under different user contexts, enabling GR models to understand and reflect multiple interpretive standards and, consequently, produce more personalized predictions.
The researchers faced two main challenges in developing Pctx. First, how to design a tokenization algorithm that can adapt based on personalized context, moving beyond the limited ‘local context’ of adjacent actions to incorporate a user’s entire interaction history. Second, how to balance the need for personalization with the generalizability that tokenization techniques typically provide. Overly personalized tokens might become too sparse, making it difficult for the model to learn and generalize.
Pctx tackles these challenges through several innovative strategies. It uses a neural model to compress the current action and a user’s interaction history into a single personalized context representation. This representation is then combined with item features and quantized into discrete tokens. This means that if two users interact with the same item for different reasons, their personalized context representations will diverge, leading to different semantic IDs for that item.
To ensure a balance between personalization and generalizability, Pctx employs adaptive clustering to group personalized context representations into a variable number of significant groups. It also merges infrequent semantic IDs into more semantically similar ones of the same item, preventing sparsity. Furthermore, a data augmentation strategy is used during training, where actions are augmented with alternative semantic IDs of the same item, enhancing data diversity and implicitly connecting different semantic IDs associated with the same items.
The impact of Pctx is significant. Experiments conducted on three public datasets demonstrated substantial improvements, with up to an 11.44% increase in NDCG@10 compared to non-personalized action tokenization baselines. This highlights Pctx’s ability to provide more accurate and relevant recommendations by truly understanding user-specific interpretations.
Further analysis revealed that Pctx is not just a simple combination of existing models but a fundamentally new paradigm. An ablation study confirmed the importance of each component, from the personalized context encoding using DuoRec to the clustering and merging strategies for semantic IDs, and the data augmentation and multi-facet generation during training and inference. A case study also visually demonstrated how the same item, like “StarCraft II,” could be tokenized into different semantic IDs depending on whether a user’s history indicated a preference for story-driven games or real-time strategy games, showcasing its ability to capture multifaceted attributes.
Also Read:
- CausalRec: Enhancing Recommendation Systems by Understanding Why Users Act
- Improving Language Model Uncertainty Estimates Through Diverse Sampling
This work marks a crucial step forward in generative recommendation, introducing the first personalized context-aware tokenizer. By allowing items to be tokenized into multiple semantic IDs based on user context, Pctx enables GR models to capture diverse user interpretations and generate more user-specific predictions. For more details, you can read the full paper here: Pctx: Tokenizing Personalized Context for Generative Recommendation.
