TLDR: This research introduces a novel framework for generating counterfactual explanations in rule-based smart environments. These explanations clarify what needed to be different to achieve a desired outcome, offering actionable insights. A user study found that while causal explanations are linguistically simpler, counterfactuals are preferred for their practical, solution-oriented content, especially when users want to resolve a problem or change a situation. The findings emphasize the importance of delivering context-aware explanations tailored to user needs and goals in smart systems.
Smart environments, like our homes and offices, are becoming increasingly automated, thanks to advancements in the Internet of Things (IoT) and Artificial Intelligence (AI). These environments use rules to make devices act autonomously, from turning on lights to adjusting thermostats. While convenient, this automation can often be confusing. Users frequently struggle to understand why a system behaved a certain way, especially in multi-user settings where rules are shared and managed.
This is where explainability comes in. Providing clear explanations helps users understand system behavior, builds trust, and improves their overall experience. Among various types of explanations, counterfactual explanations are particularly powerful. Instead of just telling you why something happened (a causal explanation), a counterfactual explanation tells you what could have been done differently to achieve a desired outcome. For example, instead of “The lamp is on because it’s after 5 p.m.,” a counterfactual explanation might say, “The lamp would have been off if the room had been empty.”
Despite their intuitive nature and potential benefits, there haven’t been established methods for generating counterfactual explanations specifically for rule-based smart environments. This research paper, titled “From Facts to Foils: Designing and Evaluating Counterfactual Explanations for Smart Environments” by Anna Trapp, Mersedeh Sadeghi, and Andreas Vogelsang, addresses this gap.
A New Framework for Understanding Smart Environments
The researchers propose the first formal definition and implementation of counterfactual explanations tailored for rule-based smart environments. Their work introduces a practical framework that extends an existing explanation engine called SmartEx.
At its core, the framework defines a counterfactual explanation as the “minimal change” required to achieve a desired alternative outcome, which they call the “Foil,” instead of the actual outcome, the “Fact.” This minimal change principle is crucial because humans prefer simple explanations that focus on the main causes.
The framework identifies three main scenarios where users might need an explanation:
- An undesired event occurred (e.g., the lamp turned on unexpectedly).
- An expected event did not occur (e.g., the speaker remained off).
- A different event occurred than what was anticipated (e.g., the blinds rolled down halfway instead of completely).
To achieve the desired “Foil,” the system employs different strategies. These can involve “additive” explanations, which suggest activating an appropriate rule, or “subtractive” explanations, which focus on inactivating a rule that caused the undesired outcome. For instance, to turn a lamp off, an additive explanation might suggest activating a rule like “If the room is empty, turn off the lamp,” while a subtractive explanation might suggest inactivating a rule like “If it is after 5 p.m., turn on the lamp.”
The framework then computes the “minimal change” by evaluating candidate explanations against five desirable properties: Controllability (can the user enact the change?), Sparsity (how few changes are needed?), Temporality (how recent was the change?), Proximity (how many other system changes would result?), and Abnormality (is the change unusual?). These properties are aggregated using a method called TOPSIS to select the best, most actionable explanation.
The final explanation is presented in natural language, using a counterfactual conditional tense, such as: “The Device of Interest would be in the Expected State if additive changes had happened and subtractive changes had not happened.”
User Preferences: Context is Key
To evaluate their approach, the researchers conducted a user study with 17 participants, comparing their generated counterfactual explanations against traditional causal explanations. The study revealed that there was no single preferred explanation type; instead, user preference was highly contextual.
Key findings include:
- Linguistic Clarity vs. Content Usefulness: Participants found causal explanations easier to understand linguistically, but they rated counterfactual explanations slightly more useful in terms of content.
- User Goals: Those who preferred shorter, solution-oriented explanations favored counterfactuals. Conversely, users who preferred longer, reason-oriented explanations leaned towards causal ones.
- Situational Context: Causal explanations were preferred in time-pressured situations, suggesting they require less cognitive effort. However, counterfactual explanations were strongly preferred when users had a high need for an explanation and a desire to change the situation.
- Actionability: Counterfactual explanations were significantly more preferred when they offered actionable advice. Subtractive counterfactuals (preventing an event) were slightly more favored than additive ones (introducing a new event).
The study highlights that intelligent systems should not rely on a single explanation method. Instead, they should adapt the type of explanation provided based on the user’s specific context and goals.
Also Read:
- Boosting Teamwork in AI: How Prompt Engineering and LLMs Enhance Collaborative Agents
- Collaborative Agents for Automated Data Visualization
Looking Ahead
This research provides a foundational framework for a new type of explanation in smart environments, offering empirical evidence to guide the choice between causal and counterfactual explanations. Future work could explore using large language models to enhance the linguistic quality of these explanations and conduct larger, in-situ validations.
You can read the full research paper here: From Facts to Foils: Designing and Evaluating Counterfactual Explanations for Smart Environments.
