TLDR: ToolCritic is a new diagnostic framework that improves the reliability of large language models (LLMs) in multi-turn, tool-augmented dialogues. It identifies eight specific types of tool-use errors, such as premature tool invocation or incorrect arguments, and provides targeted feedback to the LLM. This feedback loop enables the LLM to revise its responses, leading to up to a 13% improvement in tool-calling accuracy over baselines. The framework was trained on a synthetic dataset and demonstrated effectiveness across various LLMs, highlighting the importance of external feedback for correcting tool-use errors.
Large language models (LLMs) are becoming increasingly common in conversational applications, often needing to use external tools like booking systems or search engines to fulfill user requests. However, these models frequently make mistakes when interacting with these tools, which can hinder their reliability in real-world scenarios.
A new diagnostic framework called ToolCritic has been introduced to address these challenges. It’s designed to evaluate and improve how LLMs behave in multi-turn, tool-augmented conversations. ToolCritic can identify eight specific types of errors related to tool-calling, such as calling a tool too early, providing incorrect arguments, or misinterpreting the tool’s output. Once an error is detected, ToolCritic provides targeted feedback to the main LLM, which then uses this feedback to revise its response.
The researchers systematically defined these error categories and created a specialized synthetic dataset to train ToolCritic. This dataset was constructed by injecting single mistakes into otherwise error-free dialogues from the Schema-Guided Dialogue (SGD) dataset, using a few-shot prompting approach with the Claude 3.5 Sonnet model. Each injected error was carefully annotated with its category and a detailed explanation of why the response was incorrect.
Experiments conducted on the SGD dataset demonstrated that ToolCritic significantly improves tool-calling accuracy. It achieved up to a 13% improvement over existing methods, including zero-shot prompting and self-correction techniques. This is a crucial finding, as previous research has shown that LLMs often struggle to reliably identify their own errors, making external feedback mechanisms like ToolCritic particularly valuable.
The framework operates by inspecting the assistant’s proposed response at each turn of a dialogue. If an error is found, ToolCritic generates descriptive textual feedback. The assistant LLM, which is assumed to have strong reasoning capabilities, then uses this feedback to correct its response in real-time. This approach contrasts with self-correction methods, which often yield inconsistent results because LLMs are not adept at spotting their own mistakes.
The study evaluated ToolCritic with various LLMs, including Claude 3 Sonnet, Mistral Large 2, and LLaMA 3.1 70B Instruct. Claude 3 Sonnet showed the most substantial improvement in success rate, increasing by 13% when using ToolCritic’s full feedback. The analysis of error types revealed that “premature invocation” – calling a tool before gathering all necessary information – was one of the most common mistakes detected.
While ToolCritic shows promising results, the authors acknowledge certain limitations. The method adds to inference costs due to the additional diagnostic step and potential re-generation by the LLM. It also requires labeled training data, although the synthetic data generation pipeline helps mitigate this. Furthermore, ToolCritic is currently trained on a fixed set of tool schemas from the SGD dataset and is not designed for zero-shot generalization to entirely new tools. Future work aims to address these limitations and expand the model’s capabilities.
Also Read:
- Boosting Recommendation Accuracy: A New Approach with Language Models and Collaborative Feedback
- AI Agents Learn and Adapt Through Dialogue to Tackle Complex Problems
This research represents a significant step towards more robust integration of LLMs with external tools in real-world dialogue applications, enhancing their reliability and effectiveness in complex conversational scenarios. You can read the full paper here: ToolCritic: Detecting and Correcting Tool-Use Errors in Dialogue Systems.
