TLDR: A new research paper proposes a simple yet effective confidence-weighted averaging method to combine pre-trained state-of-the-art deep learning models for vehicle trajectory prediction. This ensemble approach improves prediction performance by 10% over the best individual model, particularly for challenging “long-tailed” scenarios, without requiring costly retraining. The method was validated on NuScenes and Argoverse datasets using models like AutoBot, Wayformer, and MTR, demonstrating significant improvements in accuracy and robustness.
Autonomous vehicles rely heavily on accurate trajectory prediction to navigate safely and efficiently in complex urban environments. As advanced deep learning models for this task continue to emerge, a significant challenge lies in effectively combining their strengths without the need for expensive retraining. A recent research paper introduces a surprisingly simple yet effective approach to enhance trajectory prediction by ensembling pre-trained models.
The paper, titled “Ensemble of Pre-Trained Models for Long-Tailed Trajectory Prediction,” by Divya Thuremella, Yi Yang, Simon Wanna, Lars Kunze, and Daniele De Martini, explores how a confidence-weighted average method can significantly boost prediction performance. This method combines the outputs of state-of-the-art deep learning models directly, without any retraining or fine-tuning, leading to a 10% improvement over the best individual model, especially in challenging “long-tailed” scenarios.
The Challenge of Trajectory Prediction
Predicting vehicle trajectories is a multidimensional regression problem. Modern methods often produce multiple plausible paths for a vehicle, but real-world applications like path planning require precise and convergent predictions. The traditional “Best-of-N” metric, which evaluates how close the best of N predicted paths is to the ground truth, can be misleading. It can reward “shotgun” predictions—widely spread, uninformed guesses. Therefore, the researchers focused on evaluating a single, “most-likely” prediction and its associated confidence, which is more practical for autonomous vehicle systems.
A Simple Yet Powerful Solution: Ensemble Modeling
The core idea is to combine the predictions of several pre-trained models. The researchers used three prominent transformer-based models: AutoBot, Wayformer, and MTR. These models vary in architecture size and training strategies, leading to different individual performances. Despite these differences, combining their outputs proved beneficial.
The proposed methodology involves a confidence-weighted average. For each prediction, the most confident trajectory from each model is extracted along with its confidence score. These confidence scores are then normalized, and the final predicted trajectory is calculated as a weighted average of the individual model predictions. This approach effectively minimizes expected trajectory error by trusting higher-confidence models more and helps mitigate the impact of individual model failures.
To understand the ensemble’s own confidence, the researchers calculate the weighted variance across the three models’ predictions. A higher variance indicates greater disagreement among models, leading to a lower confidence score for the ensemble’s prediction. This is crucial for downstream tasks, as it signals when a prediction might be less reliable.
Experimental Validation and Results
The method was rigorously tested on two widely used datasets for autonomous driving: NuScenes and Argoverse 2. These datasets provide diverse urban driving scenarios, with NuScenes including both left-side and right-side driving, and Argoverse 2 covering more cities and data.
The evaluation focused on “most-likely” metrics, specifically Average Distance Error (ADE) and Final Distance Error (FDE), and critically, on “long-tailed” metrics. Long-tailed metrics assess performance on the most difficult examples (e.g., the top 1-10% of errors), which are crucial for safety-critical operations and represent rare, challenging scenarios.
The results were compelling. On the NuScenes dataset, combining all three models (AutoBot, Wayformer, and MTR) yielded the biggest improvements, even though AutoBot individually performed worse than the others. This highlights how different models have complementary strengths, tackling different types of difficult scenarios. For instance, AutoBot, despite its smaller size, sometimes excels at predicting sharp or irregular turns that other models might miss.
On the Argoverse 2 dataset, which is much larger, the improvements were also consistent, with the three-model ensemble again showing a 10% improvement in long-tail metrics. The paper notes that in many cases where the ensemble showed significant improvement, there was general agreement among models on the vehicle’s overall direction (straight, left, or right) but high variance in the exact predicted paths. The weighted average effectively finds a consensus, leading to a more accurate result.
Also Read:
- Advancing Autonomous Driving with Large Foundation Models for Trajectory Prediction
- Enhancing Autonomous Driving Safety with Uncertainty-Aware Decision Transformers
Addressing Failure Cases and Future Directions
The researchers also identified common failure cases. These include instances where one model is overly confident but incorrect, or when models predict entirely different modes of behavior (e.g., one predicts a turn, another predicts going straight). In the latter case, the ensemble might predict a path “in between,” which could be unrealistic. However, the variance-based confidence score helps by flagging these high-disagreement scenarios with low confidence, alerting downstream systems.
This work provides a flexible and modular approach to trajectory prediction, allowing new state-of-the-art pre-trained models to be easily integrated. The code for this work is open source and available at this GitHub repository. Future work aims to automatically select the optimal set of models for an ensemble and explore knowledge distillation strategies to train a single, more efficient model that can achieve similar performance to the ensemble, addressing the computational cost of running multiple models at inference time.
