TLDR: This research introduces a deep learning U-Net model with Bayesian and quantile regression methods to quantify uncertainty in surface ozone bias predictions from the MOMO-Chem model. It demonstrates the approach’s effectiveness in North America and Europe, showing how uncertainty correlates with high bias regions and evaluating the impact of land-use data on model performance, aiming to improve air quality modeling and inform policy decisions.
Air pollution poses a significant global threat, with a vast majority of the world’s population exposed to unsafe levels. A key pollutant, surface ozone (O3), is notoriously difficult to model accurately, especially at scales relevant to human health. Traditional physics-based models often fall short in providing the precision needed for practical applications. While deep learning-based emulators have shown great potential in capturing complex climate patterns, they often lack the interpretability required for critical decision-making in policy and public health. This research addresses this gap by implementing an uncertainty-aware deep learning approach to improve surface ozone predictions.
The study focuses on enhancing the Multi-mOdel Multi-cOnstituent Chemical data assimilation (MOMO-Chem) model, a state-of-the-art framework for modeling surface ozone. Although MOMO-Chem excels at large-scale ozone estimates, it struggles with finer-scale analysis due to systematic estimation errors, or bias. This bias can stem from various factors, including atmospheric chemistry, human-caused emissions, and planetary boundary layer dynamics. Accurate and timely estimates are crucial for assessing human exposure to air pollution, and addressing these biases is a vital step.
To tackle the challenge of bias correction and provide confidence in AI-generated estimates, the researchers employed an uncertainty-aware U-Net architecture. This advanced neural network was designed to predict the MOMO-Chem model’s surface ozone residuals (bias) using two distinct uncertainty quantification (UQ) methodologies: Bayesian approximation via Monte-Carlo (MC) Dropout and a quantile-based method called Conformalized Quantile Regression (CQR). The goal was to not only predict bias but also to quantify the uncertainty associated with these predictions, thereby providing a measure of confidence to users and supporting scientific acceptance.
The research demonstrated the capabilities of these techniques in estimating regional bias for North America and Europe during June 2019. The input data for the models included a comprehensive set of 28 MOMO-Chem features, such as various chemical constituents, meteorological parameters, and radiation data. Additionally, a 51-channel input was explored, which combined MOMO-Chem features with satellite data products extracted from Google Earth Engine (GEE), including land cover and population density information. The target for the models was the surface ozone bias, calculated as the difference between MOMO-Chem’s 8-hour ozone data and ground truth measurements from the Tropospheric Ozone Assessment Report (TOAR) database, a global repository of near-surface ozone observations.
The experimental setup involved training separate models for North America and Europe using daily samples from 2005-2018, with performance tested on a held-out set from June 2019. The U-Net architecture was optimized using Adam, and models were trained for 200 epochs. MC-Dropout utilized Negative Log Likelihood loss, while CQR used Quantile loss to generate predictive intervals. The uncertainty quantification methods were evaluated by comparing RMSE scores and UQ metrics like interval length and epistemic uncertainty, particularly noting how epistemic uncertainty can be reduced with more information.
The results provided valuable insights. Regarding the impact of land-use information, the study observed small improvements in RMSE scores for both MC-Dropout and CQR models in Europe, and for CQR in North America, when using the 51-channel input. For MC-Dropout in North America, increased input features led to a decrease in averaged and maximum epistemic uncertainty, suggesting that additional information can indeed reduce uncertainty. However, in Europe, increasing input features sometimes increased epistemic uncertainty, potentially due to feature correlation causing model confusion.
Spatially, the UQ techniques showed consistency in identifying regions of high uncertainty. For instance, in North America, both MC-Dropout and CQR indicated higher uncertainty on the Eastern coast, correlating with regions of high ground truth bias and larger RMSE scores. A similar pattern was observed in the South-Eastern portion of Europe. This consistency suggests that these UQ methods can effectively pinpoint areas where the model struggles to capture bias. Further investigation into specific ground stations revealed that locations with noisy ground truth signals were harder for the model to capture, resulting in larger prediction intervals and higher RMSE, while stable signals led to easier estimations and lower RMSE.
Also Read:
- New Deep Learning Method Refines Global Air Quality Estimates
- Unsupervised Correction Method Enhances Low-Cost Air Quality Sensor Reliability
The study represents a significant step forward in applying uncertainty quantification to surface ozone bias estimation. Both Bayesian approximation and quantile regression methods demonstrated their ability to identify spatial patterns of high and low uncertainty that align with areas of high bias. While there were differences in extrapolated results outside of ground station coverage and temporal variations in uncertainty, the findings underscore the potential of these techniques. Future work aims to delve deeper into temporal UQ patterns, expand the research to a global scale, and incorporate additional UQ methods to establish a comprehensive benchmark for surface ozone bias modeling. For more detailed information, you can refer to the full research paper available here.
