TLDR: A new study introduces Rule-Based Role Prompting (RRP) to make AI agents in role-playing games better at using tools and staying in character. RRP uses specific rules like “Action-first” and “Single-shot” to prevent common errors like over-speaking or making unnecessary tool calls, significantly outperforming other prompting methods in a dialogue challenge. The research highlights that explicit, rule-based constraints are highly effective for enhancing LLM agent performance in complex interactive environments.
Large Language Models (LLMs) are increasingly used to create interactive role-playing agents, from customer service bots to non-player characters (NPCs) in games. These agents need to maintain a consistent persona while also effectively using external tools to perform actions or retrieve information. However, a common challenge is that these agents often “overspeak,” generating lengthy in-character responses, and “under-act,” failing to use tools correctly or making unnecessary calls.
A recent report, “Talk Less, Call Right: Enhancing Role-Play LLM Agents with Automatic Prompt Optimization and Role Prompting,” investigates various prompting strategies to tackle these issues. The research was conducted in the context of the API track of the Commonsense Persona-grounded Dialogue Challenge (CPDC) 2025, a benchmark designed to evaluate role-playing dialogue agents in complex settings.
Exploring Prompting Strategies
The researchers, Saksorn Ruangtanusak, Pittawat Taveekitworachai, and Kunat Pipatanakul, explored four main approaches:
- Basic role prompting: A straightforward method to assign a role.
- Human-crafted role prompting: Manual adjustments based on identified weaknesses.
- Automatic prompt optimization (APO): A data-driven approach to refine prompts iteratively.
- Rule-based role prompting (RRP): An approach integrating insights from previous attempts with novel techniques.
The Power of Rule-Based Role Prompting (RRP)
The study found that the Rule-Based Role Prompting (RRP) approach delivered the best performance. RRP incorporates two innovative techniques:
1. Character-Card/Scene-Contract (CSC) Design: This method structures each turn of the dialogue into two parts: “Voice,” which dictates how the NPC speaks, and “Action,” which lists the permissible functions and specifies when to use them. Key rules within CSC include:
- Action-first: Ensures that a function call or clarification request happens before any natural language response, preventing the agent from “chatting before calling.”
- Single-shot: Limits the agent to at most one tool call per turn, encouraging the use of composite tools to gather information efficiently.
- Schema-correct: Enforces exact parameter matching for function calls, rejecting invalid calls and forcing the agent to repair or clarify.
2. Hard-Enforced Function Calling (HEF) Prompt: This is a strict system prompt for the function-calling stage. It emphasizes that the AI has “ONLY ONE CHANCE TO CALL A FUNCTION” and should choose the “fewest tools” necessary, typically one. It also stresses filling all arguments exactly and preferring composite tools over multiple smaller ones.
Addressing Common Failure Modes
These rule-based designs directly address common problems observed in other prompting methods:
- In-character bias (chat-before-call): Agents often responded in persona before invoking a tool, leading to missed or delayed calls. The “Action-first” rule in CSC mitigates this.
- Redundant multi-calls: Weaker prompts might invoke several tools for the same item. The “Single-shot” rule and HEF hard cap prevent this by consolidating information needs.
- Parameter-key drift: Schema mismatches (e.g.,
item_namesvs.item_name) would break execution. The “Schema-correct” rule ensures precise parameter matching.
Significant Performance Gains
The RRP approach achieved an impressive overall score of 0.571, a notable improvement over the baseline score of 0.519. While other methods showed some gains, RRP provided the most balanced performance across both task-oriented dialogue and persona-aware conversation sub-tasks. This suggests that explicit, rule-based constraints are highly effective in reducing tool-calling errors and improving task execution, all while maintaining strong persona consistency.
Interestingly, the study found that RRP outperformed more complex methods like Automatic Prompt Optimization (APO), demonstrating that carefully designed rules can serve as a lightweight yet powerful “function-call controller.” The researchers are also open-sourcing their best-performing prompts and the APO tool to support future development in persona prompts. You can find the full research paper here: Talk Less, Call Right: Enhancing Role-Play LLM Agents with Automatic Prompt Optimization and Role Prompting.
Also Read:
- Bridging Legal Interpretation and AI Alignment: A Framework for Consistent AI Rules
- Experience-Driven AI Agents Excel in ML Engineering
Future Directions
While highly effective, the current work focuses on API-track, function-calling tasks. Future research could extend these principles to multi-tool planning and more open-ended role-play scenarios. The strict one-call policy, while reducing redundancy, might also limit turns that genuinely require tool composition, an area for further refinement.
