TLDR: ProCause is a new method for evaluating Prescriptive Process Monitoring (PresPM) techniques. It improves upon existing methods like RealCause by incorporating sequential models (LSTMs) and an ensemble of causal inference architectures, leading to more reliable and accurate evaluations. This is particularly beneficial for handling temporal data and when the underlying process characteristics are unknown, offering a more robust framework for assessing PresPM methods.
Prescriptive Process Monitoring (PresPM) is a rapidly evolving area within Process Mining that aims to optimize business processes by recommending real-time interventions based on event log data. Imagine a system that can suggest the best action to take during a business process, like offering a discount to a customer or escalating a managerial issue, to improve key performance indicators (KPIs) such as delivery times or customer satisfaction. While the potential of PresPM is significant, evaluating these methods has always been a major challenge.
The core difficulty lies in the absence of ‘ground truth’ outcomes for all possible intervention actions. When a specific action is taken in a real-world scenario, we only observe the outcome of that action. We don’t know what would have happened if a different action had been chosen – these unobserved outcomes are known as counterfactuals. This is a fundamental problem in Causal Inference (CI), the field that helps estimate the effects of interventions.
Historically, a generative deep learning approach called RealCause has been widely used to estimate these counterfactual outcomes, allowing researchers to evaluate new PresPM policies. However, RealCause has two key limitations. Firstly, it often overlooks the temporal dependencies inherent in process data, where the sequence and timing of events are crucial. Secondly, it relies on a single CI model architecture, TARNet, which may not always be the most effective choice across all types of datasets.
Introducing ProCause: A More Robust Evaluation Framework
To address these shortcomings, researchers have introduced ProCause, a novel generative approach designed to provide a more realistic and reliable way to evaluate PresPM methods. ProCause significantly enhances the evaluation framework by offering greater flexibility and robustness.
One of ProCause’s main advancements is its support for both sequential and non-sequential models. This means it can leverage powerful sequential models like Long Short-Term Memory (LSTM) networks, which are excellent at capturing temporal dependencies in data. This is particularly beneficial for interventions where the timing and sequence of events play a critical role.
Furthermore, ProCause integrates multiple CI architectures beyond just TARNet. It includes S-Learner, T-Learner, and, crucially, an ensemble method that combines the strengths of all three. The research indicates that this ensemble approach offers the most consistent reliability, especially when the underlying characteristics of the data are unknown. This is a significant advantage because, in real-world scenarios, it’s often impossible to know which single learner architecture would be best suited for a given dataset.
How ProCause Works
At its heart, ProCause aims to approximate the true data-generating process of a business operation. It does this by fitting generative models to real historical data to simulate counterfactual outcomes. This allows for experimental control, enabling researchers to generate outcomes for various hypothetical intervention scenarios.
Unlike RealCause, which exclusively uses MLP (Multi-Layer Perceptron) based models, ProCause allows users to choose between MLPs and LSTMs as the base model for its CI learners. For TARNet, if LSTMs are chosen, they are incorporated into the shared representation layers to capture temporal patterns, while MLPs are used in the subsequent treatment-specific layers to manage complexity.
Also Read:
- Enhancing Business Process Predictions with Integrated Temporal Logic
- Unveiling True Influence: How Causal SHAP Enhances AI Explanations
Key Findings and Implications
Through extensive research using a simulator with known ground truths and validation with real-world datasets, ProCause has demonstrated several important findings:
- The ensemble learner consistently provides the most stable and accurate evaluation approximations on average. This makes it a reliable default option when the true data-generating process is unknown.
- While MLPs tend to perform slightly better for interventions with fixed timing, LSTMs show potential for improved evaluations when temporal dependencies are present, such as interventions requiring precise timing.
- It’s important to distinguish between an evaluator’s absolute performance (how accurately it estimates individual outcomes) and its relative performance (how well it ranks different PresPM methods). An evaluator might excel at one but not the other.
- ProCause’s generated outcomes and treatment assignments align well with real-world data, confirming its practical effectiveness and realism.
In essence, ProCause offers a more robust and flexible alternative to existing evaluation methods for PresPM. By providing a choice of sequential and non-sequential models, and especially by introducing an ensemble of CI learners, it helps researchers and practitioners evaluate PresPM methods with greater confidence and reliability. This advancement is crucial for the continued development and adoption of effective process optimization strategies. You can find more details about this research in the paper: ProCause: Generating Counterfactual Outcomes to Evaluate Prescriptive Process Monitoring Methods.
