TLDR: MPFormer is a new Transformer-based framework that solves the problem of multi-objective misalignment in large-scale recommendation systems. It uses an objective-conditioned transformer, personalized target weights, and enhanced user personalization to jointly optimize multiple goals (like clicks and watch time) during item retrieval. Deployed at Kuaishou, it significantly boosts user engagement and system efficiency by dynamically adjusting recommendations based on individual user preferences and task objectives.
Modern industrial recommendation systems, like those powering popular short-video platforms such as Kuaishou and Douyin, face a significant challenge: a disconnect between how items are retrieved and how they are ultimately ranked. While ranking systems often optimize for multiple goals simultaneously (like how long you watch a video, whether you click on it, or if you convert to a purchase), the initial retrieval phase often relies on simpler, single-objective models. This “semantic gap” leads to inefficiencies and limits the overall performance of these complex systems.
Current industry solutions attempt to address this by running multiple independent retrieval systems in parallel, each focused on a single objective. However, this approach comes with a major drawback: the resources required for training and serving these models grow linearly with the number of objectives. This makes it expensive and difficult to manage, especially when dealing with many loosely related objectives.
Introducing MPFormer: A Dynamic Solution
To overcome these limitations, researchers have proposed MPFormer, an adaptive framework built on the Transformer architecture. MPFormer systematically tackles the multi-stage optimization misalignment through three innovative mechanisms:
Objective-Conditioned Transformer: This mechanism allows the system to jointly understand user behavior sequences and the specific goals of different tasks (e.g., maximizing watch time vs. maximizing clicks). It uses learnable attention modulation to create dynamic interactions between what a user has done and what the system is trying to achieve.
Personalized Target Weights: MPFormer introduces a way to dynamically adjust retrieval results based on individual user preferences. This means the system can prioritize different objectives for different users in real-time, leading to more relevant recommendations.
Enhanced User Personalization: The framework incorporates detailed user personalization information directly into its core components, improving the model’s ability to represent diverse user interests and preferences.
How MPFormer Works
The MPFormer architecture consists of dedicated “user” and “item” towers, along with a multi-objective optimization framework. The user tower is responsible for understanding a user’s historical interactions and generating multiple objective-specific embeddings. It creates personalized query vectors for each objective, incorporating demographic attributes and historical interaction sequences without relying on user IDs, which helps prevent representation collapse.
A key part of the user tower is its behavior-aware attention mechanism. Unlike traditional systems, MPFormer’s design adapts to retrieval constraints by encoding historical interactions and augmenting them with objective embeddings. It uses a parameter-sharing design within its Decoder Unit, which significantly reduces computational complexity and resource requirements while maintaining performance.
The item tower, on the other hand, generates task-specific item representations. It uses a target-decoupled MLP ensemble, meaning it has separate transformation layers for each objective. This design prevents interference between competing objectives, ensuring that each task receives an optimized and relevant item representation.
MPFormer employs a multi-objective loss function that combines components for each specific objective. It uses an extended in-batch negative sampling strategy and dynamically adjusts weights for each objective based on label distribution, preventing high-frequency objectives from dominating the learning process.
Dynamic Quota Allocation for Real-Time Serving
A standout feature of MPFormer is its novel online serving architecture, particularly its dynamic quota allocation mechanism. Instead of maintaining a single index for all items, MPFormer constructs independent Approximate Nearest Neighbor (ANN) indices for each objective. During serving, it learns task-specific weights for user-item pairs based on subsequent ranking scores. These weights are then aggregated from a user’s recent interactions to determine a personalized quota allocation for each objective. This means if a user is more likely to watch long videos, the system will allocate more retrieval quota to the “long-view” objective, ensuring a personalized candidate distribution.
Also Read:
- Tailoring Recommendations for Explored and Unexplored Items in E-commerce
- Deliberate AI Reasoning Enhances Recommender Systems with STARec Framework
Real-World Impact and Efficiency
MPFormer has been successfully integrated into Kuaishou’s short video recommendation system, which serves over 400 million daily active users. Practical deployment has shown significant improvements:
Enhanced User Engagement: Online A/B tests demonstrated a +0.426% increase in total watch time, a +0.195% increase in total app usage time, and a +0.455% improvement in average view duration.
Improved System Efficiency: Compared to traditional multi-model baselines, MPFormer achieved a 60% reduction in training costs and a 66.7% reduction in inference resource requirements. It also maintained stable P99 latency at 80 ms, comparable to single-objective models.
Better Multi-Objective Exposure: The framework led to a 21.8% improvement in multi-objective exposure rates and a 12.7% increase in recall exposure rate due to dynamic quota allocation.
The MPFormer framework offers an efficient and scalable solution for multi-objective retrieval in industrial recommendation systems. By bridging the semantic gap between retrieval and ranking, and by dynamically adapting to user preferences and task objectives, it paves the way for more intelligent and personalized content delivery. You can read the full research paper here.
