TLDR: Pinterest has introduced a novel AI framework called Decoupled Entity Representation Learning for Pinterest Ads Ranking (DERM). This system uses an upstream-downstream paradigm to create robust user and Pin embeddings from diverse data sources. Upstream models learn and regularly refresh these embeddings offline, which are then used by downstream ad ranking models for CTR and CVR predictions. Key innovations include a multi-tower architecture with an ‘overtower’ for interaction features, self-supervised learning, and a weighted moving average for embedding stability. Deployed in production, the framework has shown significant improvements in both offline and online metrics, leading to better ad personalization and performance.
Pinterest, a platform known for its visual discovery and inspiration, relies heavily on sophisticated recommender systems to deliver personalized content and advertisements. A new research paper, “Decoupled Entity Representation Learning for Pinterest Ads Ranking,” introduces a novel framework designed to significantly enhance the effectiveness of these ad ranking systems.
Modern recommender systems thrive on understanding users and items (Pins in Pinterest’s case) by representing them as low-dimensional dense vectors, known as embeddings. These embeddings are crucial for tasks like finding relevant ads and predicting user engagement (clicks) and conversions (purchases or sign-ups). However, Pinterest faces a unique challenge: its diverse surfaces (Home Feed, Search, etc.) and various ad types (clicks, conversions) lead to fragmented data and specialized models, making it difficult to share information and create holistic user and Pin representations.
To overcome these hurdles, researchers at Pinterest have developed a decoupled upstream-downstream framework. This innovative approach separates the complex task of learning rich user and Pin embeddings from the real-time ad ranking process. The ‘upstream’ models are trained on vast datasets with a wide array of user interaction signals, employing advanced architectures to capture intricate relationships. Crucially, these embeddings are not computed in real-time but are pre-calculated and regularly updated offline. This asynchronous process ensures scalability and allows the ‘downstream’ ad retrieval and ranking models to integrate these high-quality embeddings as key features for predicting Click-Through Rate (CTR) and Conversion Rate (CVR).
Key Innovations of the Framework
The new framework introduces several significant advancements. Firstly, it proposes a multi-tower model architecture. This design features distinct towers for entities like users and Pins, along with a specialized ‘overtower’ that processes interaction features. This separation prevents contextual information from ‘contaminating’ the core entity embeddings, leading to more stable and robust representations. Secondly, beyond traditional supervised learning, the framework incorporates self-supervised learning using a contrastive loss technique. This method helps the model learn more meaningful relationships between user and Pin embeddings. Lastly, to manage the evolution of embeddings over time, a weighted moving average technique is employed. This approach combines current and historical embeddings, with a greater weight assigned to past embeddings, which was found to deliver the best performance improvements.
How it Works: Upstream and Downstream
The core of the upstream model, named Decoupled Entity Representation Model (DERM), uses a multi-tower architecture. It includes a User Tower, which processes diverse user features like activity sequences, demographics, and interests. A Pin Tower handles all ad Pin-related features, such as content metadata and aggregated statistics. An Overall Interaction Tower then processes the interactions between the outputs of these two towers, along with other contextual features. These towers utilize a Deep Hierarchical Ensemble Network (DHEN) backbone. The models are trained daily on Pinterest’s large-scale ad engagement (CTR) and ad conversion (CVR) datasets. The resulting DERM embeddings are then aggregated using a moving average (with a weight of 0.8 for past embeddings) and stored for online serving.
For downstream tasks, these DERM embeddings are integrated into existing CTR and CVR prediction models. The CTR model, for instance, is a multitask learning framework that uses a Mixture of Experts (MoE) module for feature interactions. The CVR model has a similar architecture. The integration of DERM embeddings significantly enhances the predictive power of these models.
Also Read:
- Bridging Grocery and General Merchandise: A New Recommendation System for E-commerce
- LinkedIn’s AI-Powered Notification System Boosts User Sessions
Impressive Results in Practice
Extensive experiments, both offline and online, have demonstrated the effectiveness of the DERM embeddings. Offline tests showed that the weighted moving average aggregation strategy performed best, and combining user and item embeddings from both CTR and CVR upstream models yielded the highest performance gains. In online A/B tests, the framework delivered significant improvements. For CTR prediction, the treatment group showed a substantial lift in both CTR and ‘good’ CTR (gCTR, indicating longer engagement). For CVR prediction, there was a notable reduction in Cost Per Acquisition (CPA) and an increase in both click conversion rate (cCVR) and view conversion rate (vCVR).
This framework has already been deployed in Pinterest’s production ad ranking systems, leading to tangible gains in online metrics. The researchers plan to continue their work by incorporating additional data sources, exploring a single upstream model for unified embedding learning, and developing content-based embeddings to further enhance cross-domain transfer learning capabilities. You can read the full research paper for more technical details at arXiv:2509.04337.
