TLDR: Researchers have introduced SWiM (Spacecraft With Masks), a new dataset of nearly 64,000 annotated spacecraft images, created using real models, synthetic backgrounds, and noise to mimic real-world conditions. This dataset addresses the scarcity of training data for autonomous spacecraft inspection. They fine-tuned YOLOv8 and YOLOv11 nano segmentation models to generate performance benchmarks under strict hardware and inference time constraints typical of onboard flight computers. The models achieved a Dice score of 0.92, a Hausdorff distance of 0.69, and an inference time of approximately 0.5 seconds, successfully meeting NASA’s deployment requirements for real-time autonomous spacecraft inspection.
Spacecraft operating in the harsh environment of outer space are constantly exposed to various forms of damage. Repairing these issues in space, whether through human extravehicular activity or robotic manipulation, is incredibly costly and risky. This highlights an urgent need for autonomous inspection systems that can reliably analyze diverse spacecraft targets, from defunct satellites to interplanetary probes, in real-time.
Recent advancements in image segmentation, a computer vision technique that precisely outlines objects within an image, offer a promising path towards developing reliable and cost-effective autonomous inspection systems. However, a significant hurdle has been the scarcity of publicly available, annotated datasets specifically designed for spacecraft segmentation.
Introducing the SWiM Dataset
To address this critical data gap, researchers have introduced a new, comprehensive dataset called Spacecraft With Masks (SWiM). This dataset comprises nearly 64,000 annotated spacecraft images, making it the largest and most diverse of its kind to date. The images were generated using a unique dual-methodology approach:
- Existing spacecraft models were superimposed onto a mixture of real and synthetic backgrounds.
- Synthetic samples were generated using NASA’s TTALOS (Toolset for Training and Labeling in an Optical Simulator) pipeline, which integrates astrophysical backgrounds created with Stable Diffusion and procedurally rendered 3D spacecraft models.
To accurately mimic real-world conditions, the dataset also incorporates various camera distortions and noise, ensuring that models trained on SWiM are robust and generalizable to actual in-space scenarios.
Meeting Onboard Computing Constraints
A key challenge for autonomous inspection systems is the stringent hardware and computational limitations of onboard flight computers. These systems typically operate on a 4-core CPU with less than 4GB of RAM, and require an inference time (the time it takes for the model to process an image) of under 0.95 seconds for real-time performance. Traditional segmentation models often exceed these constraints.
To provide a realistic performance benchmark, the researchers fine-tuned YOLOv8 and YOLOv11 nano segmentation models. The YOLO (You Only Look Once) framework is renowned for its real-time inference capabilities and efficiency, making its nano variants ideal for resource-constrained environments like NASA’s Intel UP Board, which is used in inspector spacecraft. These models were optimized using quantization techniques and exported to ONNX format to ensure compatibility and performance on the target hardware.
Also Read:
- Real-time Landmark Tracking: A New AI Approach for Autonomous Spacecraft Navigation
- DatasetAgent: Automating Image Dataset Creation with Multi-Agent AI
Performance Benchmarks
The models were evaluated using two key metrics: the Dice coefficient, which measures the overlap between predicted and ground truth segmentation masks, and the Hausdorff distance, which quantifies the accuracy of object boundaries. The results were highly encouraging:
- Both YOLOv8 nano and YOLOv11 nano models achieved a high average Dice score of 0.92, indicating excellent accuracy in identifying the main area of the spacecraft.
- They also demonstrated very low average Hausdorff distances (between 0.69 and 1.07), showing their proficiency in detecting the complex boundaries of various spacecraft models.
- Crucially, the models achieved an approximate inference time of 0.5 seconds per image, which is nearly half the required 0.95-second constraint.
This work represents a significant step forward in developing vision-based systems that meet the demanding requirements of in-orbit autonomy. By providing a large, annotated dataset, defining clear hardware and inference time constraints, and establishing performance benchmarks, this research offers a robust framework for future advancements in real-time spacecraft inspection. The SWiM dataset and the benchmark models are publicly available, fostering further research and development in this critical area. You can find more details in the research paper.
