TLDR: LCDiff is a new framework for adverse weather image restoration that uses a Lumina-Chroma Decomposition Network (LCDN) in YCbCr color space to separate degradation-related luminance from degradation-invariant chrominance. It then employs a Lumina-Guided Diffusion Model (LGDM) with a Dynamic Time Step Loss to refine restoration, focusing on both low and high-frequency details without explicit weather prompts. The paper also introduces DriveWeather, a comprehensive dataset for evaluating AWIR in driving scenarios, demonstrating LCDiff’s superior performance over existing methods.
Images captured in adverse weather conditions like rain, haze, or snow often suffer from reduced clarity, obscured details, and unwanted artifacts. These issues significantly hinder the performance of critical vision-based tasks such as object detection and autonomous driving. Traditional methods struggle to adapt to the unpredictable nature of weather degradations, while newer approaches relying on “prompt learning” can be inconsistent if the weather conditions are not accurately estimated.
Addressing these challenges, a new framework called LCDiff has been introduced. Developed by Wenxuan Fang, Jili Fan, Chao Wang, Xiantao Hu, Jiangwei Weng, Ying Tai, Jian Yang, and Jun Li, LCDiff offers a robust solution for restoring images affected by various adverse weather conditions. The core innovation lies in its two main components: the Lumina-Chroma Decomposition Network (LCDN) and the Lumina-Guided Diffusion Model (LGDM).
Lumina-Chroma Decomposition Network (LCDN)
The LCDN is designed to process degraded images in the YCbCr color space, rather than the more common RGB. This is a crucial insight, as experiments showed that weather-induced degradations primarily affect the luminance (Y) channel, while the chrominance (CbCr) channels (which handle color information) remain largely intact, only experiencing minor blurring. By separating these components, LCDN can specifically target and mitigate degradation in the luminance channel while preserving the original color fidelity. This decomposition simplifies the restoration task, making it more effective at removing weather artifacts.
Lumina-Guided Diffusion Model (LGDM)
Even after LCDN’s initial restoration, some challenging scenarios, like excessive blurring, can still leave artifacts. This is where LGDM comes in. It leverages the partially restored luminance information as a guiding condition for a diffusion model. Diffusion models are powerful generative models that gradually remove noise from an image to reveal a clean version. By guiding this process with luminance information, LGDM eliminates the need for explicit weather prompts, making the restoration more stable and adaptable to diverse real-world degradations. This approach helps recover fine details that might otherwise be lost.
A notable feature of LGDM is its Dynamic Time Step (DTS) Loss. Previous diffusion models often prioritize restoring low-frequency features (like overall brightness and large shapes), sometimes at the expense of high-frequency details (like textures and sharp edges). The DTS Loss addresses this by dynamically adjusting its focus during the denoising process. It emphasizes low-frequency consistency early on and gradually shifts its weight to high-frequency details, ensuring a balanced recovery of both global structure and intricate textures.
Introducing DriveWeather: A New Dataset for Robust Evaluation
To thoroughly evaluate and advance adverse weather image restoration, the researchers also developed DriveWeather, a comprehensive all-weather driving dataset. Existing datasets often lack realistic atmospheric effects or focus on static objects, which isn’t ideal for autonomous driving scenarios. DriveWeather includes 29,750 precisely aligned image pairs across seven challenging weather conditions—densefog, rainfog, overcast, rainstreaks, puddles, droplets, and glare—each with five severity levels. This dataset provides a rigorous benchmark for testing AWIR systems in environments critical for autonomous navigation.
Also Read:
- Semantic Guidance for Detailed 3D Point Cloud Generation
- Optimizing Diffusion Model Training with Autoguidance and Smart Data Curation
Superior Performance
Extensive experiments demonstrate that LCDiff surpasses state-of-the-art methods across various benchmarks, including the new DriveWeather dataset. It consistently produces sharper structures and more natural textures, effectively removing artifacts while preserving details and illumination. The method shows significant improvements, particularly in complex conditions like rain and fog, and rain streaks. The effectiveness of each component, the dynamic time step loss, and the choice of YCbCr color space were all validated through detailed ablation studies, confirming their critical contributions to the overall performance.
This innovative framework marks a significant step forward in adverse weather image restoration, offering a more robust and efficient solution for enhancing visual clarity in challenging environments. For more technical details, you can refer to the full research paper here.
