TLDR: ChoirRec is a novel AI framework that uses Large Language Models (LLMs) to create semantic user groups, specifically targeting the challenge of predicting conversion rates for low-activity users in e-commerce. By generating robust semantic profiles and employing a dual-channel architecture for knowledge transfer, ChoirRec effectively mitigates data sparsity and noise. Extensive experiments on Taobao show significant improvements in conversion rate prediction, leading to a 7.24% increase in order volume for low-activity users in online A/B tests.
In the vast world of e-commerce, accurately predicting what a user might buy is crucial for a successful recommendation system. However, a significant challenge arises when dealing with ‘low-activity users’ – those who don’t interact much with the platform. These users often have sparse and unreliable behavioral data, making it difficult for traditional systems to understand their preferences and predict their conversion rates (CVR).
This is where ChoirRec, a new framework developed by researchers at Alibaba Group and Tsinghua University, steps in. ChoirRec aims to tackle these limitations by leveraging the advanced semantic capabilities of Large Language Models (LLMs) to create meaningful user groups and significantly improve CVR prediction for these underserved low-activity users. You can read the full research paper here: ChoirRec: Semantic User Grouping via LLMs for Conversion Rate Prediction of Low-Activity Users.
The Core Challenges for Low-Activity Users
Existing recommendation systems struggle with low-activity users due to three main issues:
- Noisy and Unreliable Data: Limited interactions mean less reliable signals about user preferences.
- Insufficient User Information: A lack of diverse interaction data makes it hard to build a complete picture of a user.
- Bias Towards High-Activity Users: Models are often trained on abundant data from active users, overshadowing the needs of those with fewer interactions.
How ChoirRec Works: A Three-Part Harmony
ChoirRec introduces a novel approach with a dual-channel architecture designed for robust knowledge transfer across users. It consists of three main components:
1. Semantic Group Generation
Instead of relying on raw, noisy behavioral data, ChoirRec uses LLMs to create rich ‘semantic profiles’ for each user. These profiles capture a user’s core identity, key interests, and consumption philosophy by analyzing static attributes, aggregated historical purchases, and recent search queries. These semantic profiles are then clustered into ‘semantic user groups’ using a technique called Residual Quantization K-Means. This process helps filter out noise and creates stable, reliable groups that can be used as a foundation for understanding user preferences.
2. Group-aware Hierarchical Representation
Once semantic groups are formed, ChoirRec enriches the sparse information of individual low-activity users with valuable group-level insights. This involves three aspects:
- Hierarchical Group ID Fusion: Each user’s group membership is represented as a hierarchical code, which is then fused to create a comprehensive group ID representation.
- Group Attribute Completion: If a user is missing static attributes, the model uses aggregated statistics (like the most common value or average) from their semantic group to fill in the gaps.
- Group Behavioral Sequence Construction: To address sparse purchase histories, ChoirRec identifies the most frequent interest categories within a group and constructs a representative behavioral sequence, providing a dense signal of common group interests.
3. Group-aware Multi-granularity Module
This module is crucial for ensuring that the valuable group-level knowledge is effectively integrated without being overshadowed by individual-level signals, especially from high-activity users. It uses a ‘dual-channel’ architecture:
- Individual Channel: Focuses on a user’s personal and dynamic preferences.
- Group Channel: Models the stable, shared preferences of the user’s semantic group.
These channels interact through carefully designed mechanisms:
- Asymmetric Information Injection: Group knowledge is selectively injected into the individual channel, enhancing it without compromising the stability of the group channel.
- Gated Knowledge Distillation: The individual channel (acting as a ‘teacher’) provides soft guidance to the group channel (the ‘student’), but only from reliable, high-activity users. This prevents noisy signals from negatively impacting the group channel’s learning.
Finally, the predictions from both channels are adaptively fused, with the model relying more on the group channel for low-activity users and more on the individual channel for high-activity users.
Also Read:
- Deepening Knowledge Integration: How Semantic-Condition Tuning Enhances LLMs for Knowledge Graph Completion
- TripScore: A New Benchmark for Real-World AI Travel Planning
Real-World Impact on Taobao
ChoirRec was put to the test with extensive offline and online experiments on Taobao, a leading e-commerce platform. The results were impressive:
- Offline Evaluation: ChoirRec improved GAUC (a key metric for personalized ranking quality) by 1.16% for low-activity users and 0.47% overall.
- Online A/B Testing: A 21-day live A/B test showed significant business gains, with a 7.24% increase in order volume and a 9.27% increase in Gross Merchandise Volume (GMV) specifically for low-activity users. Overall, orders increased by 2.23% and GMV by 3.10%.
These results demonstrate ChoirRec’s practical value and its ability to drive measurable improvements in core business metrics by effectively addressing the challenges of user sparsity in recommender systems.
