TLDR: Speeder is a novel framework designed to enhance Multimodal Large Language Models (MLLMs) for sequential recommendation. It addresses key challenges like suboptimal item representations, modality-related cognitive bias, and weakened sequential perception in long interaction sequences. Speeder introduces three innovations: Multimodal Representation Compression (MRC) for concise item tokens, Modality-aware Progressive Optimization (MPO) for gradual multimodal learning, and Sequential Position Awareness Enhancement (SPAE) for improved sequence understanding. Experiments show Speeder significantly boosts training speed by 250% and reduces inference time to 25% on real-world datasets, while also improving recommendation accuracy.
Sequential recommendation systems are crucial for predicting what users will interact with next, based on their past behavior. With the rise of Large Language Models (LLMs), these systems have gained powerful generative and reasoning abilities, significantly boosting their performance. Multimodal Large Language Models (MLLMs) take this a step further by incorporating diverse data types like images and interactive relationships, offering an even richer understanding of user preferences.
However, current MLLM-based recommendation systems face several significant challenges. One major issue is the suboptimal way items are represented. Long and redundant descriptions lead to inefficiencies in both training and making predictions. Another problem is a ‘modality-related cognitive bias,’ where LLMs, primarily trained on text, struggle to effectively integrate and use non-textual information like images. Lastly, in long sequences of user interactions, the models often lose track of earlier behaviors, weakening their ability to understand long-range dependencies.
To tackle these critical issues, researchers have introduced a new and highly efficient framework called Speeder. This innovative paradigm for MLLM-based sequential recommendation features three core advancements:
Multimodal Representation Compression (MRC)
Speeder addresses the problem of lengthy and redundant item descriptions by introducing Multimodal Representation Compression (MRC). This component condenses various item attributes—such as text, images, and unique identifiers—into concise yet highly informative tokens. By reducing redundancy, MRC significantly lowers computational costs during both training and inference. It uses pre-trained encoders to extract initial features and then employs a Mixture of Modality Experts (MoME) to facilitate cross-modal interaction and fusion, creating a unified, compact item embedding.
Modality-aware Progressive Optimization (MPO)
To overcome the cognitive bias LLMs have towards non-textual data, Speeder proposes Modality-aware Progressive Optimization (MPO). This strategy guides the LLM to gradually learn multimodal representations through a three-stage training process. It starts with understanding compressed text and the recommendation task, then progressively introduces visual data, and finally integrates all multimodal content. This gradual approach, along with a special ‘tanh gating mechanism,’ prevents the LLM from being overwhelmed by the sudden introduction of non-textual information, ensuring smoother convergence and better understanding.
Also Read:
- Unlocking Adaptive Recommendations: How AdaRec Uses LLMs for Personalized Experiences
- HyMoERec: A Dynamic Approach to Personalized Recommendations
Sequential Position Awareness Enhancement (SPAE)
The challenge of weakening sequential perception in long interaction sequences is addressed by Speeder’s Sequential Position Awareness Enhancement (SPAE). This module significantly improves the LLM’s ability to capture both relative and absolute sequential dependencies. SPAE achieves this through two mechanisms: a Position Proxy Task (PPT) that forces the LLM to focus on the relative order of items, and Position Prompt Learning (PPL), which injects learnable embeddings to strengthen the representation of absolute positions within a sequence. A dynamic truncation strategy also helps manage varying sequence lengths effectively.
Extensive experiments conducted on real-world datasets, including Automotive, Home & Kitchen, and Clothing & Shoes from Amazon, demonstrate Speeder’s remarkable effectiveness and efficiency. The framework not only outperforms existing MLLM-based state-of-the-art sequential recommendation models in accuracy but also delivers substantial speed improvements. Specifically, Speeder increases training speed to 250% of the original and reduces inference time to 25% on the Amazon dataset, making it a highly practical solution for real-world applications.
Speeder represents a significant step forward in making multimodal sequential recommendation more efficient and accurate, paving the way for more sophisticated and responsive recommendation systems. You can read the full research paper here: A Remarkably Efficient Paradigm to Multimodal Large Language Models for Sequential Recommendation.
