TLDR: ALGOS is a new AI system that uses language-guided vision-language models to monitor harmful algal blooms (HABs). It combines remote sensing image segmentation with severity estimation, outperforming previous methods. By integrating GeoSAM for high-quality segmentation and fine-tuning on NASA’s CAML dataset for severity, ALGOS offers a comprehensive and automated solution for tracking HABs, crucial for ecological and public health management.
Harmful algal blooms (HABs), particularly those dominated by cyanobacteria, are an increasing global concern. These blooms pose significant threats to aquatic ecosystems and human health by depleting oxygen, releasing toxins, and disrupting marine biodiversity. Traditional methods for monitoring HABs, such as manual water sampling, are often labor-intensive and limited in their ability to cover large areas or track changes over time.
Recent advancements in artificial intelligence, specifically vision-language models (VLMs) used in remote sensing, have shown promise for developing scalable, AI-driven solutions. However, a key challenge has been the ability of these systems to both interpret imagery for bloom presence and accurately quantify the severity of the blooms.
A new research paper introduces ALGae Observation and Segmentation (ALGOS), a novel system designed for comprehensive HAB monitoring. ALGOS integrates remote sensing image understanding with a robust severity estimation capability. This innovative approach combines GeoSAM-assisted human evaluation to create high-quality segmentation masks, which are then used to fine-tune a vision-language model for predicting bloom severity. The system leverages the Cyanobacteria Aggregated Manual Labels (CAML) dataset from NASA for its severity predictions.
The ALGOS framework is a unified vision-language system that bridges reasoning segmentation with HAB severity assessment using satellite imagery. It utilizes a novel HAB segmentation dataset curated through GeoSAM-assisted annotation with human evaluation. This allows the system to simultaneously localize the extent of blooms spatially and classify their severity levels through natural language reasoning.
The methodology behind ALGOS involves a comprehensive pipeline for generating high-quality pixel-level segmentation masks for HAB detection in Sentinel-2 imagery. It extends GeoSAM with an interactive mask generation and human evaluation stage, where users provide prompts to guide the model, and human annotators validate the generated masks to ensure accuracy. For severity assessment, ALGOS adopts a synthetic query generation pipeline, refining WHO recreational guidance thresholds into a five-level ordinal scale (Very low, Low, Moderate, High, Very high). Each reasoning query is paired with a satellite image and its corresponding severity label, enabling the model to learn the relationship between visual appearance, ecological context, and severity categories.
The core of the Vision-Language Model Architecture for HAB Monitoring in ALGOS follows an embedding-as-mask paradigm. It integrates domain-adapted visual encoders with language models fine-tuned on HAB-reasoning queries. The system uses a Vicuna-7B language model as its base, coupled with a Remote-CLIP ViT-L/14 encoder optimized for satellite imagery. A specialized <SEG> token triggers segmentation mask prediction via a SAM decoder head. The model is optimized end-to-end with a joint objective that combines text generation and segmentation losses.
Experimental results demonstrate that ALGOS achieves strong performance in both segmentation and severity prediction tasks. For segmentation, it significantly surpasses baseline models like LISAT and LISA-7B in both per-image class-balanced mean IoU (cIoU) and dataset-level global IoU (gIoU). In severity prediction, ALGOS substantially reduces error compared to the LLaVA-7B baseline, showing improvements in Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE).
Also Read:
- Evaluating AI’s Ability to Spot the Unknown in Plankton Monitoring
- ReefNet: A New Standard for Global Hard Coral Classification with AI
In conclusion, ALGOS represents a significant step forward in automated HAB monitoring. By integrating geospatial foundation models to jointly perform both spatial segmentation and severity estimation on wide-area remote sensing imagery, it offers a robust tool for ecological monitoring and policy support. While the current framework has been evaluated on a limited geographic and seasonal scope, future work aims to extend its evaluations and data pipelines for broader applicability. You can read the full research paper here: Seg the HAB: Language-Guided Geospatial Algae Bloom Reasoning and Segmentation.
