TLDR: SITS-DECO (Satellite Image Time Series-DECoder Only) is a new generative model for Earth Observation (EO) data, inspired by large language models. It uses a simple decoder-only architecture to process diverse EO tasks as unified token sequences, combining continuous data with symbolic instructions. The model demonstrates strong performance in multi-modal, multi-temporal crop-type classification, outperforming larger EO foundation models by effectively handling dense temporal data and simplifying pre-processing. It supports multi-task learning and promptability through symbolic tokens, offering a flexible, data-centric approach to EO modeling without requiring architectural modifications for new tasks or modalities.
A new research paper introduces SITS-DECO, a novel generative model for Earth Observation (EO) data that takes inspiration from large language models like GPT. This model aims to simplify and improve how we use satellite data for various real-world tasks, moving away from rigid, task-specific models towards a more flexible, unified approach.
Traditional machine learning models for satellite data often require significant adaptation for different tasks and are built around specific data sources. SITS-DECO, which stands for Satellite Image Time Series-DECoder Only, addresses these limitations by adopting a generative, decoder-only architecture. This means it processes all information, whether input or output, within a single, shared representational space, much like how GPT models handle text.
A Unified Approach to Earth Observation
The core idea behind SITS-DECO is to represent diverse EO tasks, including both pre-training and downstream applications, as unified sequences of tokens. These sequences combine continuous data (like satellite reflectance or backscatter values) with symbolic elements (such as task instructions or crop types). By predicting the next token in a sequence, the model implicitly learns a wide range of capabilities without needing specific architectural changes for each task or data type.
The researchers, Samuel J. Barrett and Docko Sow, demonstrated SITS-DECO’s effectiveness using pixel-level multi-temporal satellite image time series (SITS). This type of data is crucial for many applications but has been relatively underserved by current EO foundation models. SITS-DECO showed strong performance in multi-modal, multi-temporal crop-type classification, even outperforming much larger EO foundation models on the PASTIS-R dataset. This suggests that dense temporal sequence modeling is a key missing component in many existing paradigms.
Key Innovations and Capabilities
One of SITS-DECO’s significant contributions is its ‘promptability’ through symbolic task tokens. This means that once trained, the same model can perform multiple learned tasks simply by altering the input token sequence, without any fine-tuning or specialized output heads. For example, it can be prompted to classify crop types based on Sentinel-2 data, or a combination of Sentinel-1 and Sentinel-2 data, all within the same architecture.
The model also simplifies data pre-processing. It can naturally handle irregular time series, sparse observations, and multi-modal data with minimal pre-processing, reducing the complexity often associated with preparing EO data. Furthermore, adding new tasks or modalities doesn’t require architectural modifications; new capabilities can be introduced purely by including new task examples during training, highlighting a data-driven extensibility.
Experiments showed that SITS-DECO significantly outperformed existing EO foundation models on the PASTIS-R crop classification benchmark, with a gap of over 15 mIoU from the highest-performing foundation model. This highlights a persistent weakness in current spatial-first approaches that often lose fine-grained temporal signals. The model also demonstrated the ability to incorporate contextual metadata, like geographic location, to modestly improve classification performance, particularly for easily confused crop classes.
In a ‘massively multi-task’ experiment, SITS-DECO successfully performed 29 different task combinations simultaneously, showcasing its versatility. The research also explored using self-supervised tasks to mitigate geographic domain transfer challenges, showing a promising increase in performance on unseen regions, though still lagging fully supervised models.
Also Read:
- ZeroFlood: AI Model Enhances Flood Prediction in Data-Scarce Regions
- Unlocking Semantic Understanding in SAR Imagery with SARCLIP
Looking Ahead
SITS-DECO represents a conceptual bridge towards future generative EO foundation models. While this initial work deliberately omitted spatial context and full language integration to focus on validating the core idea, the framework is designed to be extensible. Future directions include scaling EO data, adding more input modalities, deeper exploration of self-supervised learning, and eventually integrating text and spatial context to build a truly general-purpose EO foundation model. This work emphasizes a data-centric modeling paradigm, where capabilities arise from the diversity and structure of training data rather than architectural complexity.
For more details, you can read the full research paper: SITS-DECO: A Generative Decoder-Only Model for Multitask Satellite Image Time Series Modelling.
