TLDR: This research proposes a framework using Large Language Models (LLMs) to qualitatively assess routes generated by Vehicle Routing Problem (VRP) solvers for last-mile delivery. By critiquing routes against policy-based criteria derived from driver feedback, LLMs can identify practical issues like crossing water bodies or pedestrian areas, achieving up to 86% accuracy. This approach offers a scalable way to improve cost efficiency, reliability, and sustainability in e-commerce logistics, especially in regions with complex addressing like India.
India’s e-commerce sector is experiencing rapid growth, but this expansion comes with significant logistical challenges, particularly in last-mile delivery. This crucial final leg of the delivery process can account for nearly half of an e-commerce firm’s operational expenses. While traditional Vehicle Routing Problem (VRP) solvers are widely used to plan delivery routes, their effectiveness in real-world scenarios is often hampered by issues like unstructured addresses, incomplete maps, and the sheer computational complexity of accurately estimating travel distances.
A new research paper, “Optimizing delivery for quick commerce factoring qualitative assessment of generated routes,” by Milon Bhattacharya and Milan Kumar, introduces an innovative framework to address these limitations. The study proposes using Large Language Models (LLMs) to qualitatively critique VRP-generated routes. This approach allows logistics operators to evaluate and prioritize delivery plans based on practical, policy-based criteria, moving beyond just distance and time metrics. You can read the full paper here.
Bridging the Gap Between Theory and Reality
The core problem with conventional VRP tools is their reliance on mathematically optimal solutions that may not translate well to real-world conditions. Factors such as navigating through pedestrian-only areas, crossing water bodies without bridges, or encountering railway lines can make a theoretically efficient route practically unfeasible or even dangerous for delivery personnel. These real-world complexities often force drivers to make on-the-fly adjustments, leading to delays and increased costs.
The researchers developed a framework where LLMs review generated routes against user-specified policies, which can be expressed in natural language. This is a significant departure from VRP solvers, where constraints must be defined using complex mathematical expressions. This flexibility allows for checking a wide variety of conditions and even integrating third-party tools like weather APIs for a more comprehensive assessment.
How the Framework Works
To illustrate their approach, the study used OpenStreetMap (OSM) data for Bengaluru, India, a city known for its complex topology. They generated 400 test routes using Google OR Tools, a popular VRP solver. Each segment of these routes was then evaluated by LLMs against four key questions derived from surveys with actual delivery riders: Does any of the routes cross a water body? Does it pass through a railway-crossing / railway line? Does it pass through a pedestrian area? Does the route pass through a park or forested area? These questions highlight common operational challenges faced by delivery personnel, which traditional VRP solvers often overlook.
Promising Results
The study found that LLMs were remarkably effective at identifying these routing issues. Open-source LLMs achieved an accuracy of 79% in detecting problems, while proprietary reasoning models performed even better, reaching up to 86% accuracy. This demonstrates that LLM-based evaluation can serve as a scalable and effective layer of assessment, providing insights that go beyond conventional distance and time metrics.
The implications of this research are substantial. By integrating qualitative assessments, e-commerce companies can significantly improve cost efficiency, enhance delivery reliability, and contribute to sustainability in last-mile logistics. It also opens avenues for developing fairer compensation mechanisms for delivery personnel by accounting for the actual complexity of their routes.
Also Read:
- TripScore: A New Benchmark for Real-World AI Travel Planning
- Sequence Variables: A New Era for Vehicle Routing in Constraint Programming
Challenges and Future Directions
While the findings are promising, the study also highlighted some limitations. Generic vision models, not specifically trained on geospatial data, sometimes struggled with specific map symbology. Smaller, quantized models also showed reduced ability to differentiate closely spaced features. Inference time remains a challenge, as even small models require specialized hardware for real-time operation. Additionally, ensuring consistent output formats from LLMs requires careful prompting and post-processing.
Despite these challenges, the research establishes a strong proof-of-concept. Future work could involve incorporating a wider range of qualitative criteria, testing with real-time operational data, and exploring how LLM feedback can directly refine and improve the route generation process itself. This study underscores the transformative potential of LLMs in bridging the gap between abstract optimization and the dynamic realities of last-mile logistics.
