TLDR: PDDL-INSTRUCT is a new framework that significantly improves Large Language Models’ (LLMs) ability to perform symbolic planning by teaching them logical chain-of-thought reasoning. It uses a two-phase instruction tuning process, including external verification feedback (detailed feedback being most effective), to enable LLMs to rigorously reason about action applicability, state transitions, and plan validity. This approach achieved up to 94% planning accuracy on benchmarks, a 66% absolute improvement over baselines.
Large Language Models (LLMs) have shown remarkable abilities across many fields, from writing code to solving complex math problems. However, their capacity for structured symbolic planning – the kind of precise, step-by-step decision-making required in formal systems like the Planning Domain Definition Language (PDDL) – has remained a significant challenge. A new research paper introduces PDDL-INSTRUCT, a novel framework designed to significantly boost LLMs’ symbolic planning skills through a method called logical chain-of-thought reasoning.
The core idea behind PDDL-INSTRUCT is to teach LLMs to think like a planner. This involves rigorously reasoning about every aspect of a plan: when an action can be applied, how the world changes after an action, and whether the entire sequence of actions leads to a valid solution. By breaking down the planning process into explicit logical inference steps, the framework enables LLMs to not only generate plans but also to self-correct their reasoning through structured reflection.
How PDDL-INSTRUCT Works
The framework operates in two main training phases:
1. Initial Instruction Tuning: In this first phase, a pre-trained LLM is trained using carefully designed prompts. These prompts present planning domains and problems along with detailed explanations of their solutions. Crucially, the training also includes examples of *incorrect* plans, complete with explanations of why they fail. This helps the model learn to identify and understand various planning errors, building a foundational knowledge of planning and the ability to logically justify action validity.
2. Chain-of-Thought (CoT) Instruction Tuning: This is the most innovative part of the approach. The LLM, now with a basic understanding of planning, generates step-by-step sequences of states and actions that represent a candidate plan. These sequences are then passed to an external verification module, VAL, which systematically checks the logical validity of each state transition based on action preconditions and effects. This external verification provides ground-truth feedback, which is vital because current LLMs often lack sufficient self-correction capabilities in reasoning.
The researchers explored two types of feedback from the VAL verifier: binary feedback (simply indicating if an action is valid or invalid) and detailed feedback (providing specific reasons for errors, such as a precondition not being met). Their findings confirmed that detailed feedback leads to more robust planning capabilities, offering explicit guidance on logical errors.
The CoT Instruction Tuning phase itself involves a two-stage optimization process. The first stage focuses on improving the quality of the logical reasoning chains, ensuring the model correctly checks preconditions, applies effects, and maintains invariants. The second stage then optimizes the overall planning performance, ensuring that the improved logical reasoning translates into accurate and valid plans. This dual focus helps the model develop both fundamental reasoning skills and strong task-specific performance.
Impressive Results
The PDDL-INSTRUCT framework was evaluated using Llama-3-8B and GPT-4 models on PlanBench, a standard benchmark for LLM planning capabilities. The results were highly encouraging. With detailed feedback and sufficient iterations, PDDL-INSTRUCT achieved planning accuracy rates of up to 94% in the Blocksworld domain, 64% in Mystery Blocksworld, and 79% in Logistics. This represents an average absolute improvement of 66% over baseline models and 35% over models that only underwent basic instruction tuning.
The detailed feedback consistently outperformed binary feedback across all domains and models, especially in more complex scenarios like Mystery Blocksworld, which involves semantically obfuscated predicates. This highlights the importance of specific error guidance in developing robust verification skills.
Also Read:
- Empowering Language Models with Advanced Planning Capabilities
- Transformers Grasp Planning Logic: Learning STRIPS World Models from Action Traces
Bridging the Gap
This work, authored by Pulkit Verma, Ngoc La, Anthony Favier, Swaroop Mishra, and Julie A. Shah, effectively bridges the gap between the general reasoning capabilities of LLMs and the logical precision required for automated planning. By teaching LLMs to rigorously reason through logical chain-of-thought and providing external verification feedback, PDDL-INSTRUCT offers a promising direction for creating more reliable and trustworthy AI planning systems. The full research paper can be found here.
