TLDR: ARTPS is an integrated system designed for autonomous planetary rovers to prioritize scientifically interesting targets on-site. It combines single-image depth estimation, multi-component anomaly fusion from image and depth cues, and a learnable curiosity score to rank candidate targets. The system enhances sensitivity to small, near-field objects, mitigates environmental nuisances like shadows and specularities, and provides explainable outputs for human operators, all while operating efficiently on constrained hardware. It has shown improved anomaly detection, depth estimation, and target ranking performance on Mars rover imagery.
In the ambitious realm of autonomous planetary exploration, rovers face a significant challenge: identifying and prioritizing scientifically valuable targets while operating under severe bandwidth and communication constraints. To address this, a new integrated system called ARTPS (Autonomous Rover Target Prioritization System) has been developed, aiming to enhance the onboard autonomy of these exploration platforms.
The ARTPS system is designed to make rovers smarter and more efficient in their scientific endeavors. It achieves this by combining several advanced techniques, including single-image depth estimation, a sophisticated multi-component anomaly fusion process, engineering-driven target localization, and a unique learnable curiosity score that intelligently ranks potential targets.
How ARTPS Works
The system operates through a modular pipeline, making it suitable for deployment on edge devices with limited computational power. Its core components work in harmony:
Input Enhancement: Before any analysis begins, the raw images captured by the rover undergo a series of enhancements. This includes resizing, noise reduction using bilateral filtering, local contrast improvement with CLAHE, and adaptive gamma correction. These steps clarify the images, making it easier to distinguish between the rover’s immediate surroundings and distant features, and helping to mitigate issues like shadows and specular reflections.
Single-Image Depth Estimation: A crucial aspect of ARTPS is its ability to infer depth from a single image. Utilizing a Vision Transformer (ViT)-style encoder-decoder architecture, the system generates a detailed depth map. This depth information is vital for understanding the 3D structure of the environment, allowing the rover to better perceive the size and distance of objects.
Multi-Component Anomaly Fusion: This is where ARTPS truly shines in identifying the unusual. It combines various signals to detect anomalies: reconstruction differences from an autoencoder (identifying what doesn’t fit a learned normal pattern), image-based cues like textures and edges (with built-in suppression for shadows and glare), and depth-based discontinuities (sudden changes in depth that might indicate an object or feature). These components are normalized and fused into a single, comprehensive anomaly map, which is then processed to identify candidate regions.
Localization and Box Merging: Once anomalies are detected, the system precisely locates them. It uses contour detection to generate bounding box hypotheses around the anomalous regions. These boxes are then refined and merged to create a compact set of candidate targets, ensuring consistent geometry and avoiding redundant detections.
Learnable Curiosity Score: This innovative component is at the heart of target prioritization. The curiosity score balances several factors: the ‘known value’ (how confident a classifier is about a target), the reconstruction difference (how anomalous it is), the density of combined anomalies, depth variance, and surface roughness. By learning non-negative weights for these components through regularized regression, ARTPS can rank targets based on their potential scientific interest, effectively guiding the rover’s attention to the most promising discoveries.
Explainability and Robustness
A key design principle of ARTPS is explainability. Operators can see numbered regions on the anomaly map, matched with detailed metrics in a diagnostic panel. This transparency builds trust and enables informed decision-making. The system also provides uncertainty indicators, especially for challenging areas like low-texture surfaces or extreme lighting conditions, drawing operator attention to potentially ambiguous detections.
The system is robust to common field nuisances such as shadows, specularities, and low-texture surfaces, thanks to its multi-component fusion approach and dedicated suppression mechanisms. It also maintains high sensitivity to small, near-field objects while preserving details in distant regions.
Also Read:
- Building Robots with Spatial Awareness: A Deep Dive into Scene Understanding and Reasoning
- Advancing Robot Autonomy in Space: A New Framework for Task and Motion Planning
Performance and Applications
Evaluated using imagery from NASA’s Curiosity and Perseverance Mars rovers, ARTPS demonstrated significant improvements over baseline methods. It achieved higher AUROC and AUPRC scores for anomaly detection, indicating better discrimination of unusual features. Depth estimation accuracy was also substantially improved, with reduced errors across various metrics. Crucially, the curiosity-score ranking showed superior prioritization, aligning more closely with expert judgments, as evidenced by higher nDCG, Spearman, and Kendall correlation scores.
Beyond its primary application in space missions, the ARTPS pipeline holds immense potential for various industrial and environmental scenarios. This includes road-surface anomaly detection, industrial inspection, environmental monitoring, and even medical imaging, particularly in situations with limited bandwidth and computational resources.
For more in-depth technical details and experimental results, you can refer to the full research paper here.
