TLDR: This research explores how multimodal Large Language Models (LLMs) can accurately predict driver yielding behavior at unsignalized intersections, a critical aspect of pedestrian safety. By integrating domain-specific knowledge, structured reasoning, and visual cues into their prompt design, the study found that LLMs like GPT-4o outperform traditional models in accuracy and interpretability. Deepseek-V3 excels in precision, while Deepseek-R1 prioritizes recall. The findings highlight LLMs’ potential for advanced traffic safety systems, despite current challenges in real-time data processing and computational costs.
Pedestrian safety is a growing concern in urban areas, with thousands of fatalities reported annually. The interactions between pedestrians and drivers, especially at unsignalized intersections, are complex and often difficult for traditional machine learning models to predict accurately. These models typically rely on fixed feature representations and can struggle with the nuanced, context-dependent reasoning required for such multifactorial interactions.
A recent research paper, titled Large Language Models for Pedestrian Safety: An Application to Predicting Driver Yielding Behavior at Unsignalized Intersections, explores a novel approach to this challenge by leveraging multimodal Large Language Models (LLMs). Authored by Yicheng Yang, Zixian Li, Jean Paul Bizimana, Niaz Zafri, Yongfeng Dong, and Tianyi Li, the study demonstrates how LLMs can extract patterns from diverse traffic data to accurately model driver-pedestrian interactions, specifically focusing on predicting whether a driver will yield to a pedestrian.
The Power of LLMs in Traffic Safety
LLMs, known for their ability to process and understand complex language, are uniquely suited for this task. They can integrate heterogeneous inputs, including textual reports, sensor data streams, and visual cues, to make accurate predictions. Multimodal LLMs, in particular, can interpret visual information like pedestrian body language, vehicle positioning, and traffic conditions alongside other data, offering a more comprehensive understanding of real-time dynamics that influence crash risk.
The researchers developed a framework that uses a novel prompt design. This design incorporates domain-specific knowledge (derived from descriptive statistics and statistical models), structured reasoning guidance (a step-by-step thinking process), and few-shot prompting (providing examples) to enable interpretable and context-aware inference of driver yielding behavior. The study utilized a dataset of over 3,000 annotated pedestrian-driver interaction instances from 18 unsignalized intersections in Minnesota, including both textual descriptions and representative images for multimodal analysis.
Key Findings and Model Performance
The study benchmarked state-of-the-art LLMs, including GPT-4o, GPT-4o-mini, Deepseek-V3, and Deepseek-R1, against traditional classifiers like Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and Neural Network (NN).
- GPT-4o consistently achieved the highest overall accuracy (91.06%) and recall (92.41%), demonstrating its balanced and reliable predictive capability across various traffic scenarios. Its multimodal input capability (text and image) proved highly effective in complex environments.
- Deepseek-V3 excelled in precision (92.06%), making it strong in applications where minimizing false positives is critical, such as collision avoidance alerts.
- Deepseek-R1 showed high recall (90.72%), indicating its strength in identifying true positive yielding events, which is crucial in safety-critical systems where missing a yielding event could have severe consequences.
- Traditional models like Logistic Regression performed well in simpler scenarios but struggled with the complexity of dynamic intersections.
- GPT-4o-mini, while computationally efficient, showed lower recall and accuracy in challenging environments, limiting its use to low-risk, resource-constrained applications.
Beyond predictive performance, the study highlighted the interpretability of LLMs. Unlike traditional models that offer coefficients, LLMs like GPT-4o can generate natural language explanations for their predictions, detailing how factors like vehicle speed, road markings, traffic flow, and environmental conditions influenced their decision. This human-aligned reasoning enhances transparency and trust in the model’s output.
Also Read:
- Bridging the Gap: How AI-Powered Behavioral Insights Improve Ridesourcing Predictions
- Real-World Cooperative Perception for Safer Lane Changes: A Hardware Deployment Study
Practical Implications and Future Directions
The research underscores the trade-offs between model performance, computational cost, and inference time. GPT-4o, while powerful, has higher computational demands. Deepseek-V3 offers efficiency with high precision, while Deepseek-R1 prioritizes recall despite longer inference times. The choice of model depends on the specific requirements of the deployment context, balancing accuracy, speed, and the tolerance for false positives or missed detections.
The authors acknowledge limitations, primarily the extensive manual preprocessing required for video data, which currently hinders real-time deployment. Future work aims to integrate automated real-time data processing pipelines and fine-tune open-source LLMs with domain-specific datasets to enhance performance, ensure data privacy, and reduce latency. These advancements are crucial for developing responsive, reliable, and scalable AI systems for early warning systems in urban traffic environments, ultimately improving pedestrian safety.
