TLDR: This paper introduces a novel data-centric method to reduce presentation bias in real-time neural query autocomplete systems. By generating synthetic prefixes from unbiased full user queries and combining them with real engagement data, the system enriches its training dataset. This approach, coupled with an optimized neural learning-to-rank model, significantly improves user engagement and generalization in a large-scale e-commerce environment while maintaining strict latency requirements.
In the fast-paced world of e-commerce and online search, Query Autocomplete (QAC) has become an indispensable feature. It helps millions of users daily by providing real-time suggestions as they type, reducing effort and guiding them to relevant queries. This not only enhances user experience but also improves the quality of subsequent search results. However, designing effective QAC systems presents significant challenges, primarily due to the need for low latency, high relevance, and the dynamic nature of user contexts.
Traditionally, QAC systems rely on linear models for re-ranking suggestions. While efficient, these models often struggle to capture the complex, non-linear interactions present in rich contextual features like device information, query semantics, session history, and temporal factors. As the feature space expands, linear models offer a limited approximation of user preferences, constraining their ability to leverage large-scale data effectively.
Addressing Presentation Bias in Autocomplete
A critical issue in training advanced neural Learning-to-Rank (LTR) models for QAC is ‘presentation bias’. This bias arises because users are more likely to engage with suggestions that are ranked higher, creating a feedback loop that reinforces existing rankings. This skews the training data, making it difficult for models to learn truly unbiased user preferences.
To overcome this, researchers at Walmart Global Technology have introduced a novel data-centric approach using ‘synthetic prefixes’. This method involves generating additional training samples by creating prefixes from full user queries observed in general search sessions where autocomplete was not active. These full queries represent unbiased user intent, unaffected by prior model suggestions. By combining these synthetic examples with actual QAC engagement data, the system achieves a balanced training dataset that mitigates feedback loops while preserving behavioral realism.
How Synthetic Prefixes Work
The process of generating synthetic prefixes is quite ingenious. First, the system estimates a data-driven distribution of prefix lengths from live QAC logs. This distribution helps determine the typical length at which users start interacting with autocomplete suggestions. For each full query from unbiased search logs, a prefix is randomly simulated based on this distribution. This simulated prefix is then fed into the QAC retrieval system to get a list of candidate suggestions. The original full query is marked as the positive sample, while other retrieved suggestions become negative samples. This approach introduces diverse and informative training signals that are not constrained by existing model outputs.
For instance, consider the full query “black leather jacket” from historical search logs. If a prefix ’black l’ is simulated, the QAC system might suggest ’black leather jacket’, ’black leather boots’, and ’black leather gloves’. The original ’black leather jacket’ is then treated as the positive example, and the others as negatives. This synthetic instance helps improve the model’s generalization without being influenced by past biases.
A Neural Ranker for Real-Time Performance
The core of this enhanced QAC system is a shallow feed-forward neural network. This architecture is specifically designed to meet the stringent latency requirements of real-time QAC, supporting high inference throughput (over 20,000 requests per minute per pod, ranking 50 candidates each). The neural ranker incorporates a rich set of features, including query popularity, seasonality, fuzzy match scores, and contextual signals like department affinity, device type, and alignment with previous user queries.
To further optimize training efficiency, the researchers introduced a task-specific simplification of the listwise loss function. This reduces computational complexity from O(n²) to O(n) by leveraging the unique QAC structure where each interaction involves only one ground-truth selection per prefix. This allows for scalable training on large datasets without compromising ranking quality.
Also Read:
- Next-Generation Recommendations: Combining LLMs, Multimodal Data, and Bias Mitigation
- Unlocking World Knowledge: The Evolution of Cross-Lingual Information Retrieval with Multilingual AI
Impact and Future Directions
Deployed in a large-scale e-commerce setting, the system demonstrated statistically significant improvements in user engagement, as measured by mean reciprocal rank (MRR) and related metrics. Online A/B testing showed a +0.92% to 1.49% increase in MRR across various platforms (desktop, mobile web, iOS, Android) compared to the existing linear baseline. Crucially, these gains were achieved while maintaining low latency, with the neural model operating well within system thresholds.
The findings indicate that synthetic prefixes not only improve generalization but also provide a scalable path toward bias mitigation in other low-latency ranking tasks, such as related searches and query recommendations. This work lays a strong foundation for applying advanced neural ranking techniques to real-time query suggestion systems, with promising avenues for continued improvement. Future work will focus on incorporating richer session-level features, refining device-specific adaptations, and exploring deeper neural architectures to further personalize query suggestions and enhance relevance. You can read the full research paper here.
