TLDR: A new method called ESCIM uses counterfactual inference to predict conversion rates more accurately. It generates hypothetical conversion labels for items users didn’t click, helping models learn from a broader dataset and overcome data sparsity and selection bias. Experiments show it outperforms existing methods in both simulated and real-world scenarios, leading to better predictions and reduced advertising costs.
Accurately predicting the conversion rate (CVR) is a critical task in many online systems, from e-commerce platforms to advertising networks. CVR measures the likelihood of a user completing a desired action, like making a purchase, after clicking on an item. However, this prediction is often challenging because conversions can only happen after a user clicks, meaning models are typically trained on a limited subset of data – only the clicked items.
This approach leads to two significant problems: data sparsity and sample selection bias. Data sparsity means there isn’t enough clicked data to train robust models, especially for less popular items. Sample selection bias occurs because the clicked data isn’t a random sample of all exposed items; it’s biased towards items users were already inclined to click. This can cause models to struggle with predicting interactions for less active users or niche items.
Previous attempts to address these issues, such as ESMM, ESCM2, and DCMT, have made progress but still face limitations. Some methods tend to overestimate CVR, others rely too heavily on scarce clicked data, and some make overly simplistic assumptions about non-clicked items.
A new research paper, titled “On Predicting Post-Click Conversion Rate via Counterfactual Inference,” introduces a novel framework called the Entire Space Counterfactual Inference Multi-task Model (ESCIM). This method aims to overcome the limitations of prior approaches by leveraging a powerful concept from causal inference: counterfactuals. The core idea is to answer a crucial question: “Would the user have converted if he or she had clicked the recommended item?” for items that were exposed but not clicked.
ESCIM achieves this by first building a structural causal model (SCM) that represents how users interact with items. This model helps understand the cause-and-effect relationships between exposure, clicks, and conversions. For items that users didn’t click, ESCIM performs a hypothetical intervention – it simulates a scenario where the user *did* click the item. Based on this simulation, it infers a “counterfactual CVR,” which is the predicted conversion rate if the click had occurred.
Once these counterfactual CVRs are predicted for non-clicked items, ESCIM transforms them into binary conversion labels (either 0 or 1). The paper proposes two main strategies for this transformation: a “max approach” and a “ratio approach.” The max approach uses the highest predicted CVR from actual clicked samples as a threshold, labeling non-clicked items with higher counterfactual CVRs as conversions. The ratio approach assigns conversion labels to a top percentage of non-clicked items, aiming to match the conversion ratio observed in the actual clicked data. These newly generated labels for non-clicked items are then incorporated into the model’s training process, allowing it to learn from a much richer and less biased dataset.
The effectiveness of ESCIM was rigorously tested through extensive experiments. In offline evaluations on public datasets like Ali-CCP and Ali-Express, ESCIM consistently outperformed state-of-the-art methods, showing an average improvement of 1.01% in CVR AUC and 1.02% in CTCVR AUC (Click-Through Conversion Rate AUC). Furthermore, an online A/B test conducted on a real-world advertising platform demonstrated even more significant gains. ESCIM improved CVR by 17.35%, CTCVR by 5.60%, and notably reduced the Cost-Per-Action (CPA) by 23.36% over a five-day period. This real-world validation highlights ESCIM’s practical benefits in optimizing advertising campaigns.
The research also explored ESCIM’s performance on “latent conversion data” – users who didn’t click during training but did during testing. ESCIM showed superior generalization capabilities in this scenario, achieving higher CVR and CTCVR AUCs compared to baselines. This indicates its robustness in predicting hidden behavioral shifts and adapting to unseen user interactions.
Also Read:
- Understanding Smart Environments: When “What If” Explanations Help Most
- Enhancing Time Series Predictions with Relevance-Aware Thresholding
In conclusion, ESCIM offers a powerful and effective solution to the long-standing challenges of data sparsity and sample selection bias in CVR prediction. By intelligently generating counterfactual conversion labels for non-clicked items, it enables models to learn from a more complete picture of user behavior, leading to more accurate predictions and better outcomes in real-world recommendation systems. You can read the full paper here: On Predicting Post-Click Conversion Rate via Counterfactual Inference.
