TLDR: SimpleVSF is a novel framework for end-to-end autonomous driving that enhances trajectory prediction and decision-making. It integrates Vision-Language Models (VLMs) to provide high-level semantic understanding and cognitive guidance during trajectory scoring and selection. By combining conventional and VLM-enhanced scorers with both quantitative and VLM-based qualitative fusion strategies, SimpleVSF achieves state-of-the-art performance in balancing safety, comfort, and efficiency, as demonstrated by its first-place finish in the ICCV 2025 NAVSIM v2 End-to-End Driving Challenge.
Autonomous driving technology is rapidly advancing, with end-to-end systems emerging as a promising way to create intelligent and robust driving policies. Unlike traditional modular approaches, end-to-end systems aim to reduce errors and information loss by optimizing the entire driving process from perception to action. However, these systems still face significant hurdles, particularly in making optimal decisions in complex or unusual scenarios and generating a diverse range of safe trajectories.
Addressing these challenges, researchers Peiru Zheng, Yun Zhao, Zhan Gong, Hong Zhu, and Shaohua Wu from IEIT Systems have introduced a novel framework called SimpleVSF (Simple VLM-Scoring Fusion). This innovative approach enhances end-to-end planning by integrating the powerful cognitive capabilities of Vision-Language Models (VLMs) with advanced trajectory fusion techniques. SimpleVSF was recognized as the leading approach in the ICCV 2025 NAVSIM v2 End-to-End Driving Challenge, demonstrating a superior balance of safety, comfort, and efficiency in autonomous driving.
How SimpleVSF Works
The SimpleVSF framework operates in several key stages:
1. Trajectory Candidates Generation: The process begins by generating a diverse set of possible driving paths. This is achieved using a diffusion-based trajectory generator, which takes the vehicle’s current state and a Bird’s-Eye-View (BEV) map of the surroundings as input. This stage produces a rich collection of candidate trajectories, or ‘anchors’, which serve as the foundation for subsequent evaluation.
2. VLM-Enhanced Scoring: A core innovation of SimpleVSF lies in its hybrid approach to scoring these candidate trajectories. It combines conventional, perception-based scorers with a new class of VLM-enhanced scorers. The VLM module processes front-view camera images and specific text instructions (like ‘left’ or ‘forward’) to generate high-level driving directives (e.g., ‘Accelerate, Right’). These abstract linguistic instructions are then converted into numerical features and integrated into the scoring process, allowing the system to understand deeper traffic intentions and common sense, leading to more informed trajectory evaluations.
3. Trajectory Fusion: After trajectories are scored, a fusion module selects the optimal path. SimpleVSF employs two distinct fusion strategies:
- Weight Fusioner: This mechanism quantitatively combines scores from multiple individual scorers using a weighted logarithmic sum. It aggregates diverse scoring aspects into a single value, with weights prioritizing critical metrics. The trajectory with the highest combined score is then selected.
- VLM Fusioner: This novel method leverages the VLM’s qualitative, semantic reasoning for a final refinement. The top-ranked trajectory from each individual scorer is simulated and visualized as front-view camera images. These images are then presented to a larger VLM, which makes the ultimate, context-aware selection for the planned trajectory.
Achieving State-of-the-Art Performance
The SimpleVSF framework was rigorously evaluated on the NA VSIM dataset, utilizing different splits for training, ablation studies, and competition submission. Its performance was primarily measured by the Extended Predictive Driver Model Score (EPDMS), a comprehensive metric that includes factors like collision avoidance, drivable area compliance, traffic light adherence, ego progress, time to collision, lane keeping, and comfort.
In the ICCV 2025 NA VSIM v2 End-to-End Driving Challenge, SimpleVSF achieved first place on the leaderboard with an overall EPDMS score of 53.06. It demonstrated perfect or near-perfect scores in critical areas such as Traffic Light Compliance (TLC), Drivable Area Compliance (DAC), and Driving Direction Compliance (DDC). The framework consistently led in ‘No at-fault Collisions’ (NC) scores across all competitors, showcasing its robustness and ability to adhere to crucial traffic rules while maintaining a superior balance across a wide range of metrics.
Ablation studies further confirmed the value of each component, particularly the VLM-enhanced scorers and the fusion strategies, in contributing to the overall performance gains. The research paper provides a detailed breakdown of these results. You can read the full research paper here.
Also Read:
- SparseWorld: A New Approach to 4D Occupancy Modeling for Autonomous Driving
- Enhancing VLM Agent Intelligence Through Explicit World Model Reasoning
Conclusion
SimpleVSF represents a significant step forward in end-to-end autonomous driving. By effectively combining a diffusion-based trajectory generator, a diverse set of VLM-enhanced scorers, and sophisticated fusion techniques, it successfully bridges the gap between traditional trajectory planning and the advanced semantic understanding offered by Vision-Language Models. This framework offers an innovative solution for creating safer, more comfortable, and more efficient self-driving systems, particularly in complex real-world scenarios.
