TLDR: Researchers from Walmart Global Tech developed a novel multi-task learning (MTL) framework for personalized product search ranking. It uniquely combines tabular and non-tabular data, leveraging a fine-tuned TinyBERT model for semantic understanding and a new scalable relevance labeling method. The model simultaneously optimizes for clicks, add-to-cart, and purchases, demonstrating significant performance improvements over traditional methods by effectively personalizing search results.
In the bustling world of e-commerce, finding exactly what you’re looking for quickly and efficiently is key to a great shopping experience. Imagine searching for a product and getting results that not only match your words but also anticipate your personal preferences and past behaviors. This is the essence of personalized product search ranking, a complex challenge that researchers at Walmart Global Tech have tackled with a groundbreaking new approach.
Traditional search models often fall short in today’s dynamic online retail environment. They typically focus on optimizing a single goal, like relevance, and struggle to handle the vast and varied types of data involved – from simple product attributes like price and brand (tabular data) to rich textual descriptions and customer reviews (non-tabular data). Balancing multiple objectives, such as predicting clicks, additions to cart, and actual purchases, while understanding complex user intent, has been a significant hurdle.
A Novel Multi-Task Learning Framework
To overcome these limitations, the researchers propose a novel multi-task learning (MTL) framework. This innovative system is designed to learn from both tabular and non-tabular data simultaneously, optimizing several related tasks at once to enhance overall relevance and personalization in product search.
At the heart of their approach is the integration of a pre-trained TinyBERT model. This powerful language model is fine-tuned to extract “semantic embeddings” – essentially, deep understandings of meaning – from text like search queries and product descriptions. By combining these semantic insights with traditional tabular features (like user demographics and product categories), the model gains a much richer understanding of how relevant a product is to a user’s query.
Another clever innovation is a scalable relevance labeling mechanism. Instead of relying on expensive and time-consuming human annotations, this system automatically generates relevance labels by combining various signals: how often a product is clicked (click-through rate), its position in the search results, and whether a click led to a purchase. This data-driven approach provides a robust and cost-effective way to understand product relevance.
The framework utilizes a Multi-gate Mixture-of-Experts (MMoE) architecture, which allows the model to efficiently learn from multiple tasks by using specialized “expert” networks and “gating” mechanisms to combine their outputs. The team experimented with different underlying “shared bottom” layers, including DCN-V2 and FT-Transformer, to process the combined data effectively.
What the Experiments Showed
The researchers conducted extensive experiments using a massive proprietary dataset from Walmart, containing millions of queries, products, and user interactions. They compared their model against several existing baselines, evaluating performance on predicting clicks, add-to-cart actions, and transactions. Key metrics included AUC-ROC (for prediction accuracy) and MRR@K (for ranking quality).
A new metric, “Personalization Degree (PD),” was also introduced to quantify how much the model truly personalizes results based on individual user data. A lower PD score indicates a higher degree of personalization, meaning user-specific features significantly influence the rankings.
The results were compelling:
- Multi-task learning consistently outperformed single-task models.
- Models incorporating TinyBERT for semantic embeddings showed superior performance.
- The combination of MMoE, FT-Transformer as a shared layer, and the TinyBERT-based cross-layer technique yielded the highest overall ranking performance, particularly for add-to-cart and transaction predictions.
- Using an “element-wise multiplication” (cross product) for query-product interactions within TinyBERT proved more effective than a simple “dot product.”
- Adding the relevance label prediction as a fourth optimization task further boosted the model’s performance across all existing tasks, especially for click prediction.
- The Personalization Degree scores confirmed that the model effectively tailors search results to individual users, with user-specific features altering approximately one in ten top product recommendations.
Also Read:
- Boosting LLM Search Efficiency Through Parallel Query Decomposition
- Optimizing E-commerce Coupons for Long-Term Revenue: Introducing ADT4Coupons
Looking Ahead
This research marks a significant step forward in personalized product search ranking, demonstrating how integrating diverse data types and multi-task learning can lead to more relevant and satisfying e-commerce experiences. The scalable relevance labeling mechanism also offers a practical solution to a common industry challenge.
Future work aims to explore even more sophisticated embedding techniques and enhance the relevance labeling process by incorporating large language models. The team also plans to integrate more dynamic, session-based user data into the architecture to further refine personalization. For more details, you can read the full research paper here.
