TLDR: A new research paper introduces Bayesian Modeling of Threshold Manipulation via Mixtures (BMTM) and its hierarchical extension (HBMTM) to accurately measure the causal effects of marketing spending thresholds. This framework models observed spending as a mixture of strategically affected and unaffected customers, providing robust estimates and quantifying uncertainty. HBMTM further allows for stable estimation of heterogeneous causal effects across customer subgroups, even with small sample sizes. Simulation studies and a real-world marketing application demonstrate its superior performance over traditional methods, revealing varied customer responses to incentives, including an anchoring effect among high-spending customers.
In the world of marketing, businesses often use spending thresholds to encourage customers to spend more. Think of credit card rewards or loyalty programs where you get a bonus for exceeding a certain spending amount. While these strategies are common, understanding their true impact on customer behavior has been a significant challenge for marketers.
Traditional methods, like Regression Discontinuity Design (RDD), are often used to estimate these causal effects. However, these methods can fall short when customers are aware of the thresholds and actively adjust their spending to qualify for rewards. This strategic behavior, known as threshold manipulation, can skew the results and lead to inaccurate conclusions about a marketing campaign’s effectiveness.
A New Approach to Causal Inference
A recent research paper, “Causal Inference under Threshold Manipulation: Bayesian Mixture Modeling and Heterogeneous Treatment Effects”, introduces a novel framework to tackle this problem. Developed by Kohsuke Kubota and Shonosuke Sugasawa, this study proposes a new Bayesian approach called Bayesian Modeling of Threshold Manipulation via Mixtures (BMTM), along with its hierarchical extension, HBMTM.
The core idea behind BMTM is to view the observed customer spending as a mix of two different groups: those who are strategically influenced by the spending threshold (the “bunching” customers) and those who are not (the “non-bunching” customers). By modeling these two groups separately and then combining them, the framework can more accurately estimate the true causal effect of the threshold, even when customers are actively trying to reach it.
How the Model Works
The BMTM framework uses a two-step Bayesian process. First, it estimates the spending patterns of customers who are not influenced by the threshold. Then, it fits a mixture model to the spending data around the threshold, combining the unaffected customer patterns with those of the strategically motivated customers. This Bayesian approach also provides a way to quantify the uncertainty of these estimates, giving marketers a clearer picture of the reliability of their findings.
A significant advancement of this research is the Hierarchical BMTM (HBMTM). This extension allows for the estimation of how the causal effects of thresholds vary across different customer subgroups (e.g., based on age, previous spending habits, or other characteristics). This is particularly valuable for marketing, as it enables more targeted and personalized strategies. HBMTM is designed to provide stable and accurate estimates for these subgroups, even when the sample size for a particular group is small, by intelligently sharing information across all groups.
Also Read:
- Rethinking AI Evaluation: A Framework for Robust Capability Assessment
- Unlocking Complex Causal Graphs with Entropic Inference
Real-World Impact and Findings
The researchers demonstrated the effectiveness of their methods through both simulation studies and a real-world marketing dataset. In simulations, BMTM and especially HBMTM consistently outperformed conventional RDD methods, providing more accurate and reliable estimates of causal effects. HBMTM showed remarkable stability and accuracy even in scenarios where the “bunching” signal (customers strategically hitting the threshold) was weak.
In a practical application, the HBMTM framework was applied to a marketing promotion with multiple spending thresholds (e.g., 30,000, 50,000, and 70,000 yen). Customers were divided into subgroups based on their spending in the previous month. The results revealed fascinating insights into heterogeneous customer behavior:
- Customers with lower or slightly above-threshold spending in the previous month showed a clear positive causal effect, meaning they increased their spending to meet the promotional thresholds. This aligns with the intuitive goal of such incentive programs.
- Conversely, customers who already spent significantly more than the thresholds in the previous month exhibited diminishing and even negative causal effects. This suggests an “anchoring effect,” where the relatively low promotional thresholds might have caused high-spending customers to reduce their expenditures, influenced by this new reference point.
These findings highlight the importance of understanding diverse customer responses to marketing incentives. The proposed BMTM and HBMTM frameworks offer a powerful tool for data-driven marketing, allowing businesses to design more effective and nuanced strategies by accurately measuring the causal impact of spending thresholds, even in the presence of strategic customer manipulation.
