TLDR: A research paper details the collaboration between JPL and Ubotica Technologies to demonstrate advanced onboard data analysis on the CogniSAT-6/HAMMER satellite. This 6U CubeSat, launched in March 2024, features a hyperspectral instrument and AI acceleration hardware, enabling real-time processing of Earth observation data. The project uses deep learning (U-Net CNNs) for applications like cloud screening, surface water detection, and thermal event identification, alongside spectral analysis algorithms (SAM, MF, RX) for mineral and vegetation mapping. Performing inference at the edge allows for rapid response to phenomena, dynamic targeting, and significant data volume reduction, thereby enhancing Earth science measurements and supporting NASA’s New Observing Strategies program. Initial models are set for flight demonstration in late 2024.
The Jet Propulsion Laboratory (JPL), in partnership with Ubotica Technologies, is spearheading a groundbreaking initiative to bring state-of-the-art data analysis directly to space. Their collaborative effort focuses on demonstrating advanced onboard inference capabilities using the CogniSAT-6/HAMMER (CS-6) satellite, a 6U CubeSat launched in March 2024.
CS-6 is equipped with a visible and near-infrared range hyperspectral instrument and specialized neural network acceleration hardware. This powerful combination allows for sophisticated data analysis to be performed at the edge – meaning directly on the satellite itself – rather than relying solely on data downlink to ground stations. This approach promises to unlock new possibilities for Earth science measurements and rapid responses to dynamic environmental events.
Why Onboard Data Analysis Matters
Performing data analysis onboard offers several critical advantages. Firstly, it enables rapid response to detected phenomena. Imagine a satellite identifying a wildfire or a volcanic eruption in real-time; this immediate insight can trigger intelligent planning for future measurements, allowing the spacecraft to self-cue for higher-resolution observations in the same overflight, a concept known as dynamic targeting. It can also cross-cue other satellites for rapid follow-up imagery. This capability is crucial for time-sensitive measurements of rare Earth phenomena that might otherwise be missed if ground analysis were required.
Secondly, onboard processing significantly reduces the volume of data that needs to be transmitted back to Earth. By identifying and filtering out unusable data or by summarizing key findings, the satellite can optimize its communication bandwidth, making Earth observation missions more efficient. This aligns with NASA’s New Observing Strategies (NOS) program, which aims to advance observation systems through innovative approaches.
Algorithms and Deep Learning at the Edge
The project focuses on deploying two main types of algorithms onboard CS-6: spectral analysis algorithms and deep neural networks, specifically convolutional neural networks (CNNs). Leveraging AI acceleration hardware for spectral analysis is a novel approach, pushing the boundaries of edge computing in space.
For image analysis, the team employs U-Net deep CNN architectures, tailored for efficient deployment on flight hardware. These models are designed to provide high-quality, quick classifications using minimal computing resources. They are trained to identify a wide range of Earth science features, including clouds, surface water extent (useful for flood monitoring), thermal events (like volcanoes and wildfires), land surface types (such as city, forest, water, cropland), and harmful algal blooms.
Spectral analysis algorithms, on the other hand, are used for applications like mineral and vegetation mapping. The paper details the use of three common methods: Spectral Angle Mapper (SAM), Matched Filters (MF), and the Reed-Xiaoli (RX) anomaly detector. These algorithms measure similarity between spectra or identify anomalous spectral signatures, providing insights into the composition of Earth’s surface.
Data and Verification
Given CS-6’s recent launch, the project utilizes a combination of existing datasets, including the USGS spectral library, and imagery from other satellites like Menut and Planetscope. Automated labeling techniques, such as the Haze Optimized Transform (HOT) for clouds and the Normalized Difference Water Index (NDWI) for surface water, are employed and then human-verified to ensure data quality.
Rigorous evaluation and verification are crucial. The U-Net models for cloud, thermal, and surface water detection have shown high accuracy, with cloud and thermal detection achieving over 97% accuracy. The models are not only evaluated for quality on ground hardware but also verified for computation, size, and runtime on the Myriad X Vision Processing Unit (VPU) onboard CS-6, ensuring they perform as expected in the space environment.
Also Read:
- A New Dataset for Advancing AI in Solar Physics and Space Weather Forecasting
- Boosting Edge AI Efficiency: A New Dataflow Minimizes Memory Traffic in Computing-In-Memory Systems
Current Status and Future Outlook
The development process involves training models, testing them on Myriad X Neural Compute Sticks, and then executing them on a flatsat testbed at Ubotica before final upload to CS-6. An initial set of CNN models for cloud screening, surface water extent, and thermal activity detection have been verified and are pending flight on CS-6 in September 2024. The spectral analysis algorithms are slated for demonstration onboard later in the fall of 2024.
This pioneering work, detailed in the research paper Demonstrating Onboard Inference for Earth Science Applications with Spectral Analysis Algorithms and Deep Learning, represents a significant step forward in Earth observation. By advancing the technology readiness level of onboard data analysis, JPL and Ubotica aim to enable deployment to future Earth-science missions, integrate with dynamic targeting, and explore multi-asset federated scheduling, ultimately enhancing our ability to monitor and understand our planet.
