TLDR: PolicyPad is an interactive system that enables domain experts to collaboratively prototype and refine LLM policies in real-time. By integrating UX design practices like rapid iteration and scenario-based testing, PolicyPad helps small groups of experts draft novel, domain-specific guidelines for AI behavior, particularly in high-stakes fields like mental health and law, leading to more responsible and safer AI systems.
As large language models (LLMs) become increasingly integrated into critical sectors like mental health and law, ensuring their behavior is safe and responsible is paramount. This often involves creating ‘LLM policies’ – sets of rules and guidelines that shape how these AI systems interact with users. However, the process of designing these policies has traditionally lacked the rapid experimentation, feedback, and collaborative iteration needed for such high-stakes applications.
Addressing the Challenge with PolicyPad
A new interactive system called PolicyPad aims to transform this process. Developed by researchers from the University of Washington, Carnegie Mellon University, and Princeton University, PolicyPad facilitates what they term ‘LLM policy prototyping.’ This approach borrows established concepts and practices from user experience (UX) design, such as heuristic evaluation, storyboarding, and rapid iteration, to enable domain experts to collaboratively draft and refine AI policies.
How PolicyPad Works
PolicyPad provides a real-time collaborative editor where policy designers can work together on policy statements. Crucially, it integrates a private sidebar for each user, allowing them to independently test how the policy-informed AI model behaves in various usage scenarios. This tight feedback loop means experts can immediately see the impact of their policy changes and iterate quickly.
Key features of PolicyPad include:
- Collaborative Editor: A shared workspace where multiple users can draft and refine policy statements in real-time.
- Scenario Gallery and Sidebar: Users can browse and open realistic user-AI conversation scenarios to understand current AI behavior. The private sidebar allows for independent experimentation with the policy-informed model.
- Interactive Scenario Widgets: Scenarios can be brought directly into the collaborative editor as interactive widgets, serving as illustrative examples and fostering shared context for discussions. Users can flag specific model responses for group attention.
- Drafting Blocks: These allow users to experiment with policy ideas within the editor without immediately affecting the model’s behavior, promoting non-disruptive iteration.
- Scenario Spotlights: A powerful feature that expands a scenario widget into a full card, visible to everyone. Here, the group can collaboratively edit the model’s response. After edits, PolicyPad can even suggest new policy statements based on the group’s revisions, helping to articulate their intentions.
- Heuristics Editor and Evaluator: PolicyPad helps groups define and track heuristics (guiding principles) for their policies. It can automatically evaluate the policy against these heuristics, highlighting areas for discussion and refinement.
- Version Tracking: The system keeps a history of policy versions, allowing designers to compare how different iterations impact model behavior across scenarios.
The Research Behind PolicyPad
The development of PolicyPad was informed by a 15-week observational study involving nine mental health experts. This study revealed critical needs: the desire for rapid experimentation with policy-informed models, the challenge of balancing high-level strategy with low-level wording, the importance of scenarios for grounding discussions, and the value of synchronous collaboration.
These observations led to the conceptualization of LLM policy prototyping, which involves reviewing scenarios, finalizing heuristics, collaboratively drafting policies, and testing these policies with a model. For more in-depth information, you can read the full research paper here.
Impact and Novelty
An evaluation study with 22 domain experts in mental health and law demonstrated PolicyPad’s effectiveness. The system significantly enhanced collaborative dynamics and led to the creation of more novel policies compared to a baseline system. A remarkable 51.9% of policy statements drafted in PolicyPad were considered novel, compared to just 18.2% in the baseline.
The novelty stemmed from several key areas:
- When to Defer to Humans: Policies provided specific guidance on when the AI should hand off to a human expert, especially in complex or high-risk situations (e.g., when behavioral interventions are needed in mental health, or when confidential legal information is involved).
- Specific Procedural Guidelines: Experts crafted detailed instructions, such as using motivational interviewing techniques in mental health (but avoiding them in high-risk scenarios), or clearly disclosing confidentiality limits and knowledge cut-off dates.
- Proactive Information Seeking: Policies encouraged the model to actively seek crucial information at the start of a conversation, like a user’s legal jurisdiction or details for a thorough risk assessment in mental health.
Also Read:
- UserRL: A Framework for Developing AI Agents That Truly Understand and Assist People
- AI Assistants That Listen In: Exploring the World of Overhearing LLM Agents
Looking Ahead
PolicyPad offers a promising path for integrating expert knowledge into AI alignment and safety efforts. It provides a structured yet flexible environment for small groups to develop nuanced, domain-specific policies. The researchers envision policy prototyping as a continuous process, regularly seeking expert input to evolve LLM policies based on real-world usage and emerging concerns. This work paves the way for more transparent, legible, and ultimately safer AI systems, especially in sensitive domains where responsible AI behavior is critical.
