TLDR: A new research paper introduces ‘counterfactual scenarios’ for automated planning, a novel explanation paradigm that identifies minimal changes to a planning problem itself (rather than just an existing plan) to ensure that resulting plans meet specific, desired properties. These scenarios, categorized as existential (at least one plan satisfies a property) or universal (all plans satisfy a property), address limitations of traditional counterfactual explanations by focusing on higher-level problem characteristics. The research demonstrates that generating these explanations is often computationally comparable to finding a plan, making the approach practically viable for debugging and improving AI planning systems.
Artificial Intelligence (AI) is increasingly used in critical decision-making, from credit risk analysis to bail approvals. As AI systems become more complex, understanding how they arrive at their decisions, especially in sequential tasks, becomes crucial. This need has fueled the field of Explainable AI (XAI).
In the realm of Automated Planning, where AI agents devise sequences of actions to achieve goals, existing explanation methods often focus on explaining individual decisions or reconciling differences between an AI’s model and a human’s understanding. A popular technique in XAI, called Counterfactual Explanations (CEs), shows how a minimal change to an input would lead to a different, desired output. In planning, this has typically meant identifying minimal changes to an existing plan to satisfy a different goal.
However, these traditional counterfactual explanations for planning have a limitation: they are great for diagnosing faults in a specific plan or understanding local properties, but they often fail to capture broader characteristics of the planning problem itself. For instance, they can’t easily answer questions like, “What fundamental aspect of the problem needs to change for a certain type of plan to become possible or impossible?”
Introducing Counterfactual Scenarios
To address this gap, a new research paper, Counterfactual Scenarios for Automated Planning, proposes a novel explanation paradigm based on “counterfactual scenarios.” Instead of modifying an existing plan, these scenarios identify the minimal changes needed to the *planning problem itself* such that it admits plans that comply with specific, desired properties. These properties are defined using a logical formula called LTLf (Linear Temporal Logic on finite traces), which can express complex requirements about how a plan should unfold over time.
The paper explores two main types of counterfactual scenarios:
- Existential Counterfactual Scenarios: These identify minimal modifications to a planning problem so that it admits *at least one* plan satisfying the desired properties.
- Universal Counterfactual Scenarios: These identify minimal modifications to a planning problem such that *all* its valid plans satisfy the desired properties.
Real-World Relevance: The Food Delivery Example
Consider a food delivery service with a driver, a truck, and locations like a depot, butchery, and coffee shop. If the driver starts at the coffee shop and the truck at the depot, and there’s no way for the driver to reach the truck, the problem is unsolvable. Traditional counterfactuals, which require an existing plan, would be useless here.
An existential counterfactual scenario, however, could highlight that simply changing the driver’s initial location from the coffee shop to the depot would make the problem solvable. This provides a crucial insight into why the original problem was unsolvable and how to fix it.
Another example: imagine a user wants to ensure that in some plans, the truck visits the coffee shop *before* delivering to the butchery. If the current road network doesn’t allow this direct route, an existential counterfactual scenario could suggest minimally changing the preconditions of the ‘drive’ action to allow a link between the coffee shop and the butchery, thus enabling plans with this specific sequence.
For universal requirements, such as ensuring that *all* plans eventually return the truck to the depot after all deliveries are made, a universal counterfactual scenario might suggest strengthening the goal condition to explicitly include the truck being at the depot, thereby forcing all valid plans to incorporate this step.
Also Read:
- New AI Planning Method Learns General Goals from Past Solutions
- Efficient Planning for Deterministic Multi-Agent Systems
Practical Viability and Future Directions
The researchers investigated the computational complexity of generating these counterfactual scenarios. A surprising and encouraging finding is that, for many cases, producing these explanations is often only as computationally expensive as computing a plan for the original problem. This suggests that the proposal is practically viable and lays the groundwork for developing efficient algorithms in this area.
The types of changes considered for generating scenarios include modifications to the initial state, the goal conditions, or the preconditions of actions. The framework also allows for defining “plausibility constraints” using LTLf, ensuring that the suggested changes are realistic and actionable.
This work offers a fresh perspective on explainable planning, moving beyond local plan modifications to provide prescriptive and prospective insights into how problem formulations themselves can be altered to achieve desired plan behaviors. Future work will focus on developing efficient algorithms and extending this framework to other planning formalisms.
