Improving E-commerce Search for New Products with Semantic IDs

TLDR: SMILE is a new method that enhances how e-commerce platforms represent new, “cold-start” items to improve their visibility and click-through rates. It uses a two-step encoding process (RQ-OPQ) to combine shared semantic information with unique item features and collaborative signals. This approach significantly boosts buyer numbers and order volume, especially for cold-start items, as validated by extensive offline and online testing.

In the fast-paced world of e-commerce, connecting users with the right products is crucial. However, a significant challenge arises with “cold-start” items – new products that lack sufficient user interaction data. These items often struggle to gain visibility, leading to a phenomenon known as the Matthew effect, where popular items become even more popular, while new ones remain undiscovered. This issue not only impacts platform diversity but also hinders the potential of new products.

Traditional methods attempt to address this by aligning an item’s descriptive content with its collaborative information (like user clicks and purchases). The idea is to transfer insights from well-known items to new ones. However, these approaches often fall short because they don’t fully account for the differences between content and collaborative data, nor do they capture the unique, fine-grained characteristics of individual items.

To overcome these limitations, researchers from Kuaishou Inc. and Kuaishou Technology have introduced a novel approach called SMILE: SeMantic Ids Enhanced CoLd Item Representation for Click-through Rate Prediction in E-commerce SEarch. This method aims to enhance how cold-start items are represented, making them more discoverable and improving their click-through rates.

How SMILE Works

SMILE’s core innovation lies in its use of RQ-OPQ encoding, a technique inspired by Kuaishou’s OneSearch framework. This encoding method models item features from two perspectives: hierarchical semantics (broader categories) and unique features (specific details).

The process involves two main steps:

1. Adaptive Information Transfer via RQ Encoding: SMILE uses Residual Quantization (RQ) encoding to capture the shared semantic information of items. It then designs an adaptive mechanism to transfer collaborative signals (user interaction data) from existing item IDs to these RQ encodings. This transfer is bidirectional and adapts based on whether an item is “warm” (has enough data) or “cold” (lacks data). For cold items, it injects collaborative signals obtained through convergent transfer in RQ into their IDs, while preserving their semantic information.
2. Enhancing Item Differentiated Information with OPQ Encoding: After RQ encoding, there’s residual information that represents the unique, fine-grained features of each item. SMILE uses Optimized Product Quantization (OPQ) encoding to capture this differentiated information. To further enhance this, a contrastive learning strategy is employed. This strategy helps the model learn to distinguish between similar and dissimilar items based on their unique attributes, ensuring that each item’s individuality is accurately represented.

By combining these two modules, SMILE creates item representations that are rich in both content and collaborative information, while also accurately reflecting the distinctiveness of individual items.

Also Read:

• Boosting E-commerce Search Accuracy: A Framework for Transferring LLM Reasoning to Lightweight Models
• HoMer: A Unified Approach to Enhance Click-Through Rate Prediction in Recommender Systems

Impressive Results

The effectiveness of SMILE was rigorously tested through comprehensive offline experiments on large-scale industrial datasets and real-world online A/B tests. The results were highly encouraging:

• Offline Performance: SMILE consistently outperformed existing state-of-the-art methods across all scenarios (all items, warm items, and cold-start items) in terms of AUC and GAUC metrics, which measure prediction accuracy. Its improvement was particularly significant for cold-start items, demonstrating its ability to effectively address the challenge of scarce collaborative signals.
• Online A/B Testing: In a 14-day online A/B test conducted in an e-commerce search scenario, SMILE showed statistically significant improvements. It increased the overall number of buyers by 1.72% and the order volume by 2.23%. More impressively, for cold-start scenarios, the number of buyers increased by 3.512%, and the order volume surged by 9.639%.

These results confirm SMILE’s superiority in enhancing item representations for click-through rate prediction, especially for cold-start items, making it a significant advancement in e-commerce search and recommendation systems. For more technical details, you can refer to the full research paper.