TLDR: ClearFairy is an AI assistant that helps creative professionals, especially UI/UX designers, capture and document their decision-making processes. It uses a framework called Clear to break down workflows into ‘cognitive decision steps’ by linking explanations, actions, and design artifacts. ClearFairy detects weak explanations, asks clarifying questions, and infers rationales, significantly improving the quality of documented decisions without increasing cognitive burden. The captured data also enhances generative AI agents, leading to better next-action predictions and more coherent design outcomes.
Understanding how creative professionals make decisions is crucial for learning, collaboration, and sharing knowledge. However, current methods often miss important details or leave the reasons behind decisions unclear. To tackle this, researchers have introduced a new approach called Clear, a framework that breaks down creative thinking into understandable ‘cognitive decision steps’.
A cognitive decision step is essentially a unit of decision-making that links together a designer’s actions, the artifacts they create, and their own explanations of why they did what they did. This framework helps make the entire decision-making process traceable and easier to interpret.
Building on the Clear framework, a new AI assistant named ClearFairy has been developed. This assistant is designed for UI/UX designers and works like a ‘think-aloud’ partner. As a designer works, ClearFairy observes their process. If a designer’s explanation for a decision is unclear or incomplete, ClearFairy steps in to ask lightweight, clarifying questions. What’s more, it learns from the designer’s previous explanations and can even infer missing reasons for future decisions, which the designer can then easily confirm or adjust.
A study involving twelve creative professionals showed impressive results. With ClearFairy, the quality of explanations for decision steps significantly improved, going from 14% ‘strong explanations’ to over 83%. Participants found that 85% of the rationales inferred by ClearFairy were accurate and accepted, which helped reduce the mental effort of constantly explaining their work. The system also helped designers articulate ‘implicit decisions’ – those choices that felt so natural they wouldn’t normally think to explain them, such as decisions based on common design conventions, aesthetic intuition, or personal preferences.
The captured decision steps proved useful in several ways. For instance, when these steps were provided to a Large Language Model (LLM), it was better at predicting a designer’s next actions, aligning more closely with how professionals would proceed. The documented steps also helped generative AI agents in Figma create more structured, detailed, and professionally aligned design outcomes compared to agents that only received a basic task prompt.
The researchers behind ClearFairy have also released a dataset of 417 cognitive decision steps, offering a valuable resource for future research into human knowledge-grounded creative AI agents. This work highlights the potential of structuring and capturing human decision-making to enhance learning, improve collaboration, and develop more intelligent and human-aligned AI assistants in creative fields. You can read the full research paper here: ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference.
How ClearFairy Works
ClearFairy operates through a sophisticated technical pipeline. It starts by segmenting a designer’s workflow into cognitive decision steps, using their verbal explanations, action logs, and screenshots of the design artifact. An evaluation module then checks if the explanations are sufficient. If not, a question generation module creates clarifying questions. Over time, a rationale inference module learns from the designer’s responses to predict rationales for new decisions, reducing the need for constant questioning. Finally, a documentation pipeline organizes these structured decision steps into a searchable format, including the decisions, actions, rationales, progression, and any Q&A interactions.
Also Read:
- AgentCompass: Enhancing Reliability in Production Agentic AI Workflows
- PromptSculptor: Automating Text-to-Image Prompt Refinement with a Multi-Agent System
Impact on AI Agents
The detailed cognitive decision steps captured by ClearFairy offer rich information that can significantly benefit generative AI agents. This data can help agents better understand a user’s unique workflow and personal preferences, enabling them to provide more accurate suggestions or even proactively perform actions. By incorporating these fine-grained human decision steps, AI agents can move beyond high-level reasoning to execute more deliberate, precise, and context-sensitive actions, leading to more coherent and professionally aligned creative outputs.
