TLDR: This research explores lightweight machine learning models, including gradient boosting methods (XGBoost, LightGBM) and convolutional neural networks (CNNs), for real-time cloud and cloud shadow masking directly on hyperspectral imaging satellites. The study found that all evaluated models achieved high accuracy (over 93%), with CNNs reaching over 95%. Crucially, the models were designed to be very small in size and computationally efficient, making them suitable for the limited resources of on-board satellite systems. The CNN with feature reduction (1DJuLiNetSingularityF30) emerged as the most balanced option, offering high accuracy, low storage, and fast inference, demonstrating the potential for AI-driven data processing directly in space.
Satellite remote sensing plays a vital role in observing our planet, aiding in everything from environmental monitoring to disaster response. However, a persistent challenge in optical satellite imaging is the presence of clouds, which can obscure valuable data. To ensure that only high-quality, unobstructed information is used for analysis, a crucial preprocessing step called cloud masking is required.
Traditionally, the complex computations involved in AI-based cloud masking were performed on the ground. This meant that raw satellite data, often including cloudy sections, had to be transmitted back to Earth for processing. This process is resource-intensive, requiring significant bandwidth and storage. Recent advancements in on-board processing, including lightweight deep learning models and specialized hardware like FPGAs (Field-Programmable Gate Arrays) and VPUs (Vision Processing Units), are now making it possible to perform AI-driven cloud detection directly on satellites.
The Challenge of On-Board Processing
Operating AI models directly on satellites, especially smaller CubeSats, presents unique challenges. These satellites have very limited power generation capabilities, typically around 5-8 watts. Payloads for on-board processing, such as FPGAs and VPUs, can demand around 12 watts, highlighting the need for extremely energy-efficient models. This research focuses on developing such lightweight models to enable satellites to deliver ‘analysis-ready’ data directly from orbit, reducing the need for extensive data downlink.
Exploring Lightweight AI Approaches
This study evaluated several machine learning approaches for on-board cloud masking in hyperspectral satellite imagery. Hyperspectral imaging captures detailed spectral information across numerous bands, offering a richer dataset but also increasing the complexity of cloud detection, as clouds can have diverse spectral signatures that overlap with bright surfaces like snow or sand.
The models investigated included gradient boosting methods like XGBoost and LightGBM, known for their strong performance and efficiency. Additionally, various convolutional neural networks (CNNs) were explored, including a baseline 1D CNN model (1DJuLiNet), a compressed version (1DJuLiNetSingularity), and several feature-reduced CNNs (1DJuLiNetSingularityFx, with ‘x’ indicating the number of features like 30, 18, 7, or 4). The feature reduction was achieved using principal component analysis (PCA) to reduce the high dimensionality of the hyperspectral data.
The models were trained and evaluated using a publicly available hyperspectral dataset from the HYPSO-1 mission, which provides labeled images captured by a CubeSat. This dataset is crucial for developing and testing cloud masking algorithms suitable for real-world satellite applications.
Key Findings and Performance
The experimental results were promising: all boosting and CNN models achieved classification accuracies exceeding 93%, with CNN models reaching over 95% accuracy. This demonstrates their strong capability in identifying clouds. Beyond accuracy, the study focused on model size, training efficiency, and inference speed – critical factors for on-board deployment.
All models proved to be remarkably lightweight, with trainable parameters ranging from a mere 12 to about 4500, and storage requirements between 5 kB and 265 kB. For instance, the 1DJuLiNetSingularityF07 model was exceptionally small, requiring only 5 kB of storage and having just 12 trainable parameters.
In terms of speed, XGBoost was the fastest on CPUs (240 ms per image) and efficient on GPUs (50 ms per image). While the original CNNs were slower on CPUs (4-5 seconds per image), the feature-reduced CNNs significantly improved CPU performance. Notably, the 1DJuLiNetSingularityF30 model reduced CPU inference time to 800 ms per image, making it a viable option for low-power scenarios. On GPUs, most CNN variants achieved very fast inference times, around 1 ms per image after the first image.
Training efficiency also varied, with boosting models being the quickest to train (around 12 minutes). CNNs took longer (27-50 minutes), but the 1DJuLiNetSingularityF30 model improved training efficiency, completing in 16 minutes.
Also Read:
- Boosting Agricultural Insights: Adaptive AI for Hyperspectral Image Classification
- MoSAiC: Enhancing Land Cover Classification in Remote Sensing with Hybrid Contrastive Learning
The Best Balance for Space
Considering accuracy, model size, training efficiency, and inference speed, the 1DJuLiNetSingularityF30 model emerged as the most balanced choice. It maintained high accuracy (over 94%), had a small storage footprint (20KB), required only 597 trainable parameters, and offered significantly improved inference speed on CPUs while remaining fast on GPUs.
This research confirms that lightweight AI models are a practical solution for real-time cloud and cloud shadow masking in hyperspectral imagery. These models perform consistently well with high accuracy and low computational cost, even under the strict constraints of satellite hardware. While the study used a specific dataset and did not involve actual deployment on a live satellite system, it lays strong groundwork for future in-orbit validation. For more technical details, you can refer to the full research paper. Read the full research paper here.
