TLDR: OmniAcc is an AI-powered navigation system using GPT-4o, satellite imagery, and OpenStreetMap to identify and map wheelchair-accessible features like ramps and crosswalks. It offers personalized route planning and real-time navigation, achieving 97.5% accuracy in crosswalk detection through zero-shot learning and specialized prompt engineering, aiming to make urban environments more inclusive by addressing incomplete and outdated accessibility data.
Navigating urban environments can present significant challenges for individuals with ambulatory disabilities, particularly wheelchair users. The lack of reliable and up-to-date information about accessible routes and features often forces extensive pre-planning or leads to unexpected barriers. Imagine a college student having to change their major because the building for their desired field was completely inaccessible. This highlights a critical need for better solutions.
Introducing OmniAcc: Your Personalized Accessibility Assistant
A new research paper introduces OmniAcc, an innovative AI-powered interactive navigation system designed to address these very issues. Developed by Siddhant Karki, Ethan Han, Nadim Mahmud, Suman Bhunia, John Femiani, and Vaskar Raychoudhury from Miami University of Ohio, OmniAcc aims to make urban navigation seamless and inclusive for everyone. You can read the full research paper here: OmniAcc: Personalized Accessibility Assistant Using Generative AI.
At its core, OmniAcc leverages the power of advanced generative AI, specifically OpenAI’s GPT-4o model, combined with satellite imagery and OpenStreetMap data. Its primary goal is to identify, classify, and map crucial wheelchair-accessible features such as ramps and crosswalks within the built environment. But it doesn’t stop there; OmniAcc offers personalized route planning, real-time hands-free navigation, and instant responses to user queries about physical accessibility.
How OmniAcc Works: AI with a Vision
One of OmniAcc’s most impressive aspects is its use of zero-shot learning and customized prompts. This means the system can accurately detect accessibility features without needing extensive pre-labeled training data for every single feature. It learns to recognize patterns and characteristics directly from the imagery, guided by intelligent prompts. This approach ensures precise detection and supports validation through structured workflows.
The researchers focused on crosswalk detection as a case study, achieving an remarkable accuracy of 97.5%. This high level of precision demonstrates the transformative potential of AI in improving navigation and fostering more inclusive urban spaces. Unlike traditional methods that often rely on large, manually labeled datasets, OmniAcc’s zero-shot approach makes it highly scalable and adaptable to new features like tactile paving, access ramps, or even identifying broken sidewalks.
The Technology Under the Hood
OmniAcc integrates high-resolution GeoTIFF images from the USGS EarthExplorer NAIP dataset with OpenStreetMap (OSM) data. OSM, a crowd-sourced database, provides valuable information about urban features, but it can often be incomplete or outdated. OmniAcc addresses this by automatically detecting and updating path features, ensuring the data is current and reliable.
The methodology involves several key steps:
- Data Collection and Image Patch Generation: Large satellite images are divided into smaller 256×256 pixel patches, with each road orientation within a patch treated as a separate example.
- Data Pre-processing: The researchers experimented with four different dataset variations to optimize performance:
- Plain Dataset: Raw satellite imagery.
- Separated Dataset: Raster images and vector road network data were kept separate.
- Overlaid Dataset: Road network overlays from OSM were directly superimposed onto satellite images, providing crucial directional context.
- Blurred Dataset: Non-essential regions of the images were blurred to highlight roads and crosswalks, minimizing background noise. This configuration yielded the highest accuracy.
- Prompt Engineering: Iterative refinement of prompts for the GPT-4o model, moving from basic instructions to more specific details about crosswalk locations and patterns, significantly enhanced detection accuracy.
Also Read:
- CogGuide: A Human-Inspired Approach to Smarter Multimodal AI Reasoning
- Crafting Game Worlds from Stories: An AI Approach to 2D Scene Generation
Impressive Results and Future Directions
The experimental evaluation showed that the ‘Blurred Dataset’ configuration achieved the best performance, with an F1 score of 97.53% and an accuracy of 97.5%. By focusing the model’s attention on relevant features, blurring non-essential areas proved highly effective. This success highlights the importance of targeted preprocessing in refining model reliability and accuracy.
Looking ahead, the OmniAcc team plans to expand the system’s functionality to cater to a wider range of ambulatory disabilities and to detect and map accessible features on a larger scale. Future work will also involve incorporating real-time user feedback to further refine prompts and enhance classification accuracy, as well as exploring domain-specific fine-tuning of GPT-4o or lightweight model alternatives for optimized performance. OmniAcc represents a significant step towards creating more inclusive and accessible urban environments for everyone.
