TLDR: HoMer is a novel Transformer-based model that improves click-through rate (CTR) prediction by addressing three key challenges: feature, context, and architecture heterogeneities. It achieves this through a ‘panoramic sequence’ for fine-grained user interest, a ‘set-wise’ prediction paradigm for cross-item interactions, and a unified encoder-decoder architecture for efficiency and scalability. Experiments show significant improvements in AUC, CTR, RPM, and GPU resource savings.
Click-through rate (CTR) prediction is a fundamental task in modern recommender systems, influencing everything from user engagement to platform revenue. It involves estimating the likelihood of a user clicking on a particular item, often by analyzing their past behavior and various item characteristics. However, despite significant advancements in deep learning recommendation models, researchers have identified persistent challenges that limit their effectiveness.
Understanding the Challenges in CTR Prediction
The recent research paper, “HoMer: Addressing Heterogeneities by Modeling Sequential and Set-wise Contexts for CTR Prediction,” identifies three key forms of “heterogeneity” that hinder the performance of existing CTR prediction models:
- Feature Heterogeneity: This occurs because user behavior sequences (like past clicks) are often represented by a limited number of “side features” (e.g., item category, price). In contrast, non-sequential features (like user profiles or item details) are much more extensive. This imbalance leads to a less granular understanding of user interest from sequences, impacting prediction accuracy.
- Context Heterogeneity: When a user views a set of items (e.g., on a search results page), their decision to click on one item can be influenced by the presence of other items. Traditional “point-wise” prediction models, which evaluate each item in isolation, fail to capture these crucial cross-item interactions. This misalignment with real-world user behavior limits their predictive power.
- Architecture Heterogeneity: Industrial CTR prediction models often evolve by integrating many specialized network modules (e.g., for sequence modeling, feature interaction, or enhancement). This fragmented approach can lead to overlapping functionalities, reduced efficiency, and difficulties in scaling the model effectively.
Introducing HoMer: A Homogeneous-Oriented Transformer
To tackle these limitations, researchers from Meituan have proposed a novel solution called HoMer, which stands for Homogeneous-Oriented Transformer. HoMer is designed to model both sequential user behavior and the context of entire item sets in a computationally efficient manner. It achieves this through a unified and streamlined architecture.
How HoMer Addresses Each Challenge
HoMer introduces several key innovations:
- Aligning Features with Panoramic Sequence: To combat Feature Heterogeneity, HoMer creates a “panoramic sequence.” Instead of limited side features, each behavior in this sequence incorporates all relevant non-sequential features (user/item profiles, cross features, context features) from the historical request. This provides a much richer and more fine-grained representation of user interest, significantly improving sequence modeling.
- Shifting to Set-wise Prediction: HoMer moves away from the traditional point-wise prediction paradigm to a “set-wise” approach. This means that instead of predicting clicks for each item individually, HoMer processes an entire set of items (e.g., all items on a display page) in a single model invocation. This allows the model to explicitly capture cross-item interactions, understanding how items influence each other within a given context. This parallel prediction also leads to greater computational efficiency.
- Unified Encoder-Decoder Architecture: To resolve Architecture Heterogeneity, HoMer adopts a unified encoder-decoder Transformer architecture. The encoder focuses on extracting detailed user interest from the panoramic sequence, while the decoder handles cross-item and user-item interactions. This structural simplification and shared computation enhance the model’s effectiveness, efficiency, and scalability, making it easier to maintain and deploy in large-scale industrial settings.
Also Read:
- Enhancing E-commerce Recommendations for Infrequent Shoppers with AI User Grouping
- Bridging AI Learning Methods: A New Approach to Scalable Task Adaptation
Practical Benefits and Impressive Results
Beyond its architectural innovations, HoMer offers significant practical advantages. It reduces storage costs by constructing one offline sample per request instead of per item, and it saves data I/O during training. By efficiently consuming the entire training corpus, HoMer learns more precise user behavior patterns. Online prediction services are also simplified, as HoMer’s single-pass processing eliminates the need for complex deduplication mechanisms.
Extensive offline and online experiments conducted in Meituan’s search advertising scenario demonstrate HoMer’s superiority. It outperformed the industrial baseline by 0.0099 in the Area Under Curve (AUC) metric, a significant improvement in recommender systems. Online A/B tests showed a 1.99% lift in Click-Through Rate (CTR) and a 2.46% increase in Revenue Per Mille (RPM). Furthermore, HoMer achieved a 27% reduction in online GPU resource consumption through preliminary engineering optimizations, validating its efficiency and practicality.
This innovative approach to CTR prediction, detailed in the research paper available at arXiv.org, represents a substantial step forward for industrial recommender systems, promising more accurate predictions and more efficient operations.
