TLDR: TaoSR1 is a new framework that optimizes Large Language Models (LLMs) for e-commerce search relevance. It addresses challenges like latency and reasoning errors by using a three-stage approach: Supervised Fine-Tuning with Chain-of-Thought (CoT) for reasoning, Direct Preference Optimization (DPO) for accuracy, and Group Relative Policy Optimization (GRPO) to reduce errors. It also introduces Cumulative Probability Tiering (CumPT) for simplified online deployment. The model shows significant improvements in understanding complex queries and delivering more relevant search results.
E-commerce platforms like Taobao and Amazon serve millions of users daily, making it crucial for their search engines to accurately predict the relevance between a user’s query and a product. This prediction is fundamental to a good user experience and directly impacts user satisfaction and merchant efficiency. Historically, search engines have relied on models like BERT, which are good at text matching. However, these models often struggle with more complex, long-tail queries that require deeper semantic understanding and reasoning.
Recent advancements have seen Large Language Models (LLMs) being explored for search systems. While LLMs offer superior understanding and reasoning capabilities, their direct deployment in online systems faces significant hurdles. These include high latency due to Chain-of-Thought (CoT) reasoning, the risk of errors accumulating in longer reasoning chains, and a phenomenon called ‘discriminative hallucination,’ where the model might reason correctly but still give a wrong final answer.
Introducing TaoSR1: A New Approach to E-commerce Search Relevance
Researchers from Alibaba, Fudan University, and Tsinghua University have proposed a novel LLM-based optimization framework called TaoSR1 (Taobao Search Relevance Model v1) to address these challenges. This framework is designed for direct online deployment of LLMs in e-commerce search, aiming to enhance query-product relevance prediction. You can read the full research paper here: TaoSR1: The Thinking Model for E-commerce Relevance Search.
TaoSR1 operates through a three-stage optimization process:
Stage 1: Supervised Fine-Tuning (SFT) with Chain-of-Thought (CoT)
The first stage focuses on imbuing the LLM with strong reasoning abilities. Unlike traditional methods that simply predict a label, TaoSR1 uses a generative training approach where the model directly generates the relevance label. To incorporate complex, domain-specific business rules (e.g., how to rate ‘Mate50 Pro’ for a ‘Mate50’ query versus vice-versa), the model is trained with Chain-of-Thought (CoT) reasoning. This involves a structured five-step process: Query Understanding, Product Comprehension, Category Matching, Attribute Matching, and Relevance Class Determination.
To make this practical, a Retrieval-Augmented Generation (RAG) pipeline is used. This system dynamically retrieves relevant ‘atomic’ business rules from a knowledge base and injects them into the prompt during CoT sample synthesis. This ensures the model internalizes nuanced business logic without overwhelming it with a massive rule set.
Crucially, the researchers found that a ‘respond-then-think’ approach (predicting the label first, then generating the CoT) significantly outperformed the conventional ‘think-then-respond’ method. This ‘post-CoT’ strategy mitigates error accumulation that can occur in longer reasoning paths, making the model more robust for classification tasks and feasible for online deployment.
Stage 2: Offline Multiple Sampling with Direct Preference Optimization (DPO)
Even with CoT, there’s room for improvement. The second stage uses Direct Preference Optimization (DPO) to refine the model’s generation quality. This is based on the observation that if a model generates multiple responses, it often produces at least one correct answer, even if its single-pass accuracy is lower. This ‘pass@N’ accuracy suggests that the model has the potential to solve more cases.
DPO constructs a preference dataset in two ways: For cases where the model produced at least one correct response (solvable cases), a correct response is ‘chosen’ and a random incorrect one is ‘rejected.’ For ‘hard cases’ where the model consistently failed, a more powerful ‘oracle’ model (DeepSeek-R1) is used to generate correct responses, which are then ‘chosen’ and paired with the TaoSR1 model’s incorrect ‘rejected’ responses. This dual approach helps the model learn from its successes and gain knowledge from a stronger external source for difficult problems.
Stage 3: Difficulty-based Dynamic Sampling with Group Relative Policy Optimization (GRPO)
The final stage further optimizes the model using Group Relative Policy Optimization (GRPO) combined with a difficulty-aware dynamic sampling strategy. GRPO allows for more diverse exploration of reasoning paths, helping to alleviate ‘discriminative hallucination’ – where the reasoning is correct but the final answer is not. The dynamic sampling focuses learning on instances where the model’s accuracy is neither too high nor too low, ensuring efficient training by concentrating on challenging but solvable cases.
The researchers discovered that a more balanced distribution of labels in the GRPO training data leads to superior performance. By curating a balanced subset of data, the model achieved better results even with less data, and significantly reduced the incidence of discriminative hallucination.
Simplified Online Deployment with Cumulative Probability Tiering (CumPT)
For practical online deployment, e-commerce search systems often categorize relevance into tiers (e.g., Good, Mid, Bad). Traditional methods for this tiering require multiple hyperparameters, making calibration complex and computationally expensive. TaoSR1 introduces Cumulative Probability Tiering (CumPT), a novel method that simplifies this process significantly. CumPT requires only a single hyperparameter for multi-tier assignment, streamlining deployment and providing more consistent and predictable performance by bridging the gap between offline potential and online efficacy.
Also Read:
- Boosting Search System Training with Smarter Synthetic Data Generation
- New Approach to Recommender Systems: L3AE Combines LLMs and Linear Models
Demonstrated Impact
Experimental results show that TaoSR1 significantly outperforms baseline methods on challenging offline evaluation datasets, particularly for queries requiring deep semantic understanding like negation, affordable alternatives, and knowledge-based questions. Online human evaluations also demonstrated substantial improvements in metrics like GSB (Good/Same/Bad), Query Goodrate, and Item Goodrate. The most notable gains were observed for queries seeking ‘affordable alternatives,’ a classic challenge for traditional term-matching models.
In conclusion, TaoSR1 offers a robust and innovative framework for optimizing and deploying LLMs in e-commerce search relevance. By combining structured CoT reasoning, multi-stage reinforcement learning, and a simplified tiering mechanism, it effectively balances advanced reasoning capabilities with the practical demands of real-time online systems, providing valuable insights for applying LLMs to other classification tasks.
