TLDR: A new deep learning system enables accurate indoor navigation using only a mobile phone’s camera, without GPS or special sensors. It features a graph-based method for generating diverse training paths and uses explainable AI with data augmentation and curriculum learning to improve robustness, especially in crowded environments. An Android app has been developed and tested successfully.
Navigating large indoor spaces like shopping malls or subway stations can often be a confusing experience, especially without reliable GPS signals. A new research paper introduces an innovative deep learning approach that aims to simplify indoor navigation using only the camera of a standard mobile device, eliminating the need for special sensors, pre-installed markers, or detailed scene maps.
The paper, titled “Inside Knowledge: Graph-based Path Generation with Explainable Data Augmentation and Curriculum Learning for Visual Indoor Navigation,” presents an efficient, real-time, and easily deployable system. Developed by Daniel Airinei, Elena Burceanu, and Marius Leordeanu, this method predicts the direction a user should take towards a specific target destination based solely on visual input from a smartphone.
Smart Data Generation for Training
One of the core challenges in developing such a system is acquiring sufficient training data. It’s impractical to manually record every possible path within a large building. To overcome this, the researchers devised a novel graph-based path generation method. This technique automatically creates new, diverse training paths by intelligently combining segments of real video footage. By representing intersections as nodes and corridors as edges in a topological map, the system can generate virtually any custom path, significantly enriching the training dataset and making the model more robust.
Learning from Visual Cues
The system employs a convolutional neural network (ConvNet) that processes three consecutive RGB frames from the mobile device’s camera, along with an encoded image representing the target destination. The network then predicts one of eight discrete direction classes, such as ‘forwards,’ ‘left,’ or ‘right.’ This vision-only approach means the system doesn’t rely on external information like GPS, depth sensors, or internet access during navigation.
Making the System Robust: Handling Distractions
A common issue in real-world indoor environments is the presence of moving people, which can obstruct the camera’s view and confuse navigation systems. The researchers tackled this by incorporating advanced learning strategies, including explainable data augmentation and curriculum learning. They utilized the Grad-CAM algorithm to identify which parts of an image the model focuses on when making decisions, particularly when it makes errors. This revealed that people close to the camera were often the cause of misdirection.
To mitigate this, they introduced several data augmentation techniques during training. One method, ‘PeopleMask,’ involved automatically detecting people in training videos and replacing them with random noise, forcing the model to learn from static environmental features. Another, ‘GradMask,’ intelligently masked regions that Grad-CAM identified as critical for the model’s decisions, effectively making the training process harder and the model more resilient to visual distractions. These augmentation strategies, combined with curriculum learning (where more challenging training cases are introduced later), significantly improved the system’s accuracy, especially in crowded scenes.
Also Read:
- Advancing GUI Automation with Difficulty-Aware Reinforcement Learning
- Autonomous Rovers Master Challenging Lunar Landscapes
Real-World Application and Future Prospects
The researchers collected a unique, large-scale dataset by recording video footage in a shopping mall over different times of the year and with varying crowd levels. The true ground truth direction for each frame was automatically estimated using a vision-only method combining optical flow and monocular depth estimation.
The system has been implemented as an easy-to-use Android application and successfully tested in real-time. The tests demonstrated its ability to guide users accurately to their destinations, with the system quickly correcting any rare errors. Furthermore, the method proved robust even when tested in ‘out-of-distribution’ scenarios, such as navigating during different seasons with altered decorations, highlighting its adaptability.
This work marks a significant advancement in mapless indoor navigation, offering a practical, affordable, and easily deployable solution for consumer-grade mobile devices. The authors plan to make their application, dataset, and training code publicly available, paving the way for further research and broader adoption. For more details, you can refer to the original research paper: Inside Knowledge: Graph-based Path Generation with Explainable Data Augmentation and Curriculum Learning for Visual Indoor Navigation.
