TLDR: Align2Act is a new autonomous driving framework that uses instruction-tuned large language models (LLMs) to create interpretable and human-aligned motion plans. It breaks down driving decisions into a step-by-step reasoning chain, incorporating human logic and traffic rules, and generates both trajectories and their rationales. Evaluated on real-world benchmarks, it shows improved planning quality and human-likeness, making self-driving decisions more transparent.
Autonomous driving faces a significant challenge in motion planning, especially in complex and unpredictable environments. Traditional methods, whether rule-based or learning-based, often struggle with adaptability, robustness, and providing clear explanations for their decisions. This is where the innovative Align2Act framework steps in, aiming to bridge the gap by transforming large language models (LLMs) into interpretable and human-aligned planners for self-driving cars.
The core idea behind Align2Act is to leverage the powerful reasoning capabilities of LLMs, but with a crucial difference: it explicitly incorporates human reasoning patterns and traffic rules into the planning process. Instead of just generating trajectories, Align2Act guides LLMs through a step-by-step reasoning process, producing not only the vehicle’s path but also the rationale behind that path. This makes the system far more transparent and understandable, addressing a key concern in autonomous vehicle development.
How Align2Act Works
Align2Act formulates motion planning as a language generation problem. It takes a structured textual input that describes the vehicle’s context, including environmental observations, its current state, and specific planning instructions (like “turn right” or “yield near intersection”). From this, the LLM generates both the desired trajectory and a detailed reasoning trace, called the Align2ActChain.
The Align2ActChain is central to its interpretability, breaking down the decision-making into four distinct stages:
- Preliminary Planning: Identifies a high-level maneuver, such as continuing in a lane or preparing for a turn.
- Collision Prediction: Forecasts the movement of other vehicles and pedestrians, flagging potential hazards.
- Traffic Context Assessment: Considers external factors like traffic light states, speed limits, and lane boundaries.
- Final Action Integration: Synthesizes all this information to determine the safest and most appropriate driving action, which is then translated into a continuous trajectory.
This structured approach ensures that the model’s decisions are not black-box outputs but are grounded in understandable logic, much like how a human driver would reason through a situation.
Instruction-Based Alignment and Model Architecture
To ensure the model learns human-aligned behavior, Align2Act uses imitation learning with prompt-based supervision. This means the model is trained with natural language prompts that describe scenarios, intended maneuvers, and constraints, effectively teaching it to “think” like a human driver. The Align2ActDriver framework uses LLaMA-2-7B as its base, fine-tuned efficiently using Low-Rank Adaptation (LoRA) to adapt it for motion planning without requiring massive computational resources.
Also Read:
- Flow Planner: Advancing Autonomous Driving with Intelligent Interaction Modeling
- Unlocking AI’s Decisions: A New Approach to Explainable Driver Intention Prediction
Real-World Evaluation and Performance
The researchers evaluated Align2Act on the nuPlan dataset, a comprehensive collection of real-world autonomous driving scenarios. Unlike many prior works that focus on synthetic or open-loop settings, Align2Act was tested on the nuPlan closed-loop benchmark (Test 14-random and Test 14-hard), which simulates real-time interaction with dynamic environments. The results showed improved planning quality and human-likeness, with Align2Act achieving strong Open-Loop Scores (OLS) and competitive Closed-Loop Scores (CLS) compared to traditional rule-based, hybrid, and even some learning-based planners.
Ablation studies further confirmed the importance of the structured reasoning chain and scenario diversity for robust performance. While Align2Act demonstrated significant advancements in interpretability and human alignment, the paper also acknowledges current limitations, such as its performance in closed-loop settings still lagging behind some conventional planners, and the computational demands of LLMs. Future work aims to integrate visual inputs, reduce latency, and scale to broader benchmarks.
This research represents a significant step towards creating autonomous driving systems that are not only capable but also transparent and trustworthy, by making their decision-making process understandable to humans. For more details, you can read the full research paper here.
