TLDR: IndiaWeatherBench is a new, open-source dataset and benchmark for regional weather forecasting specifically for the Indian subcontinent. It uses high-resolution reanalysis data (IMDAA), provides standardized training and evaluation, and includes various machine learning models (UNet, Transformers, Graph-based networks) with different boundary conditioning strategies. The benchmark highlights the importance of architectural design and conditioning strategies, especially during extreme weather events, aiming to accelerate research in regional weather prediction.
Regional weather forecasting is crucial for localized climate adaptation, disaster mitigation, and sustainable development. While machine learning has made significant strides in global weather prediction, regional forecasting has remained relatively underexplored. This is partly due to a lack of standardized datasets and consistent experimental setups, which makes fair comparisons and reproducibility challenging.
To address this gap, researchers have introduced IndiaWeatherBench, a comprehensive benchmark designed specifically for data-driven regional weather forecasting over the Indian subcontinent. This initiative aims to provide a unified framework for the machine learning community to advance regional weather prediction research.
IndiaWeatherBench offers a carefully curated dataset built from high-resolution regional reanalysis products. It includes a suite of deterministic and probabilistic metrics, ensuring consistent training and evaluation across different models. The benchmark also establishes strong baselines by implementing and evaluating a range of machine learning models, including UNets, Transformers, and Graph-based networks, along with different strategies for handling boundary conditions and various training objectives.
The Indian subcontinent was chosen as the region of interest due to its immense societal relevance, with over 1.4 billion people whose lives depend on weather-sensitive sectors like agriculture and water management. Furthermore, India presents unique scientific challenges for forecasting models, given its extraordinary climatic diversity, from arid deserts to tropical rainforests and monsoon coasts. These heterogeneous and dynamic weather patterns are difficult for coarse global models to capture accurately.
The dataset within IndiaWeatherBench is based on the Indian Monsoon Data Assimilation and Analysis (IMDAA) reanalysis dataset. IMDAA provides hourly observations at a high spatial resolution of approximately 12km, specifically tailored to Indian monsoon dynamics. IndiaWeatherBench offers a preprocessed version of IMDAA, covering 20 years of multi-channel atmospheric states at 6-hour intervals, with standardized train-validation-test splits. It includes 43 distinct meteorological variables, such as temperature, wind components, precipitation, and pressure, at various atmospheric levels.
The benchmark supports two primary data formats: Zarr for scientific analysis and HDF5, which is optimized for machine learning applications, allowing for efficient loading and batching of data.
Regional weather forecasting in IndiaWeatherBench is framed as learning a function that maps historical regional weather states and auxiliary information to future forecasts. A key aspect explored is how models handle atmospheric dynamics outside the regional domain, known as boundary conditioning. Two strategies are investigated: boundary forcing, which uses high-resolution data at the spatial boundaries, and coarse-resolution conditioning, which integrates lower-resolution global forecasts from systems like ERA5.
The baseline models evaluated include UNet, a convolutional neural network; Stormer, a transformer-based architecture; and Graph-based models like GraphCast and its hierarchical extension, Hi. These models are trained using either deterministic prediction (minimizing latitude-weighted mean squared error) or probabilistic modeling (using denoising diffusion models to capture uncertainty).
Experimental results show that with boundary forcing, Stormer and GraphCast generally achieve the best performance across most variables and lead times. However, under coarse-resolution conditioning, Stormer’s performance significantly degrades, likely due to an incompatibility between its input tokenization scheme and the mixed spatial scales of the input. This highlights the critical importance of aligning architectural design with the chosen boundary conditioning strategy.
The benchmark also evaluates model performance during extreme weather events, such as a heatwave in May 2019. During this event, the Hi model demonstrated the most realistic forecasts, closely matching ground truth, while Stormer and UNet showed a warm bias, and GraphCast exhibited a cold bias. This indicates that models can behave substantially differently under rare conditions.
Also Read:
- Enhanced Precipitation Forecasting: The Advanced Torrential Loss Function
- Unlocking Deeper Insights: A New Approach to Forecasting Multivariate Time Series
IndiaWeatherBench is fully open-sourced, including all raw and preprocessed datasets, model implementations, and evaluation pipelines. This promotes accessibility, transparency, and reproducibility, fostering broader community participation in atmospheric science from the machine learning domain. While currently relying on ground-truth auxiliary inputs, future work aims to incorporate real-time global forecasts and expand to more regional domains. For more technical details, you can refer to the original research paper.
