TLDR: LIDAR is a novel lightweight neural network designed for efficient and accurate pixel-level segmentation of structural cracks using various types of multimodal data, including RGB, infrared, polarization, and depth images. It features a Lightweight Adaptive Cue-Aware Visual State Space module (LacaVSS) for adaptive crack cue modeling, a Lightweight Dynamically Modulated Multi-Kernel convolution (LDMK) for reduced computational cost, and a Lightweight Dual Domain Dynamic Collaborative Fusion module (LD3CF) for effective multimodal feature integration. Experiments show LIDAR outperforms existing methods in performance and efficiency, making it suitable for resource-constrained environments.
Structural cracks are a common issue in materials like asphalt, concrete, and masonry, often leading to significant losses if not detected and monitored regularly. Traditional methods for identifying these cracks often rely on single-modality data, such as standard RGB images, which can be unreliable under varying lighting conditions or in the presence of background noise. These methods also struggle to capture crucial information like subsurface thermal anomalies or spatial hierarchies found in depth images.
Recent advancements in deep learning, particularly with Convolutional Neural Networks (CNNs) and Transformers, have shown promise in multimodal semantic segmentation. However, CNNs can have limited receptive fields and high computational costs due to numerous operations. Transformers, while effective at capturing long-range dependencies, suffer from quadratic scaling of attention mechanisms, making them computationally expensive for high-resolution images and unsuitable for deployment on devices with limited resources.
Introducing LIDAR: A Novel Approach to Crack Segmentation
To overcome these challenges, researchers have proposed a new network called LIDAR, which stands for Lightweight Adaptive Cue-Aware Vision Mamba. This innovative network is designed to efficiently perceive and integrate morphological (shape) and textural cues from various data types, including RGB images, infrared thermography, polarization information, and light-field depth cues. The goal is to generate clear, pixel-level crack segmentation maps with low computational cost, even with arbitrary input sizes.
LIDAR achieves its impressive performance and efficiency through two main components: the Lightweight Adaptive Cue-Aware Visual State Space module (LacaVSS) and the Lightweight Dual Domain Dynamic Collaborative Fusion module (LD3CF). Additionally, it introduces a Lightweight Dynamically Modulated Multi-Kernel convolution (LDMK) that replaces most traditional convolutional operations, significantly reducing computational overhead.
How LIDAR Works: Key Innovations
The **LacaVSS module** is responsible for adaptively modeling crack cues. It incorporates an Efficient Dynamic Guided Scanning Strategy (EDG-SS). Unlike previous methods that use fixed scanning rules, EDG-SS dynamically prioritizes crack regions based on image content, using pre-scanned masks. This not only accelerates the generation of scanning sequences but also improves the efficiency of texture modeling and the accuracy of distinguishing cracks from the background.
The **LDMK convolution** is a crucial innovation for reducing computational cost. It dynamically selects the most important feature channels for processing, avoiding redundant computations. This module also uses an adaptive selective kernel strategy to capture morphological cues across multiple receptive fields while maintaining low complexity. This design allows LIDAR to achieve excellent performance with a minimal number of parameters.
For effective cross-modal and hierarchical feature fusion, LIDAR employs the **LD3CF module**. This module includes an Adaptive Frequency Domain Perceptron (AFDP) that enhances high-frequency crack features (which represent fine details and edges) and suppresses low-frequency background noise. Combined with a dual-pooling fusion strategy and dynamic gating, LD3CF enables efficient multi-level interaction between different data modalities at a low computational cost.
Also Read:
- Bridging the Data Gap: How Physics-Informed Transfer Learning Enhances Structural Health Monitoring
- RaGS: Advancing 3D Object Detection with 4D Radar, Monocular Cameras, and Gaussian Splatting
Performance and Efficiency
Extensive experiments were conducted on three datasets: IRTCrack (RGB and infrared), CrackDepth (RGB and light-field depth), and CrackPolar (RGB and various polarization images). LIDAR consistently outperformed other state-of-the-art methods across various dual-modal and multi-modal input combinations. For instance, on the light-field depth dataset, LIDAR achieved high F1 and mIoU scores with remarkably few parameters.
In terms of computational efficiency, LIDAR demonstrated significantly lower FLOPs (floating-point operations), parameter counts, and model sizes compared to other leading methods. This efficiency is a direct result of the lightweight LDMK design, LacaVSS’s adaptive modeling, and LD3CF’s ability to suppress noise while enhancing key features.
The research paper, available at https://arxiv.org/pdf/2507.22477, concludes that LIDAR represents a significant step forward in multimodal structural crack segmentation. Its ability to efficiently extract and fuse diverse cues from multiple modalities, coupled with its low computational cost, makes it a highly promising solution for real-world crack detection applications. Future work aims to further enhance LIDAR’s adaptability to modality-specific differences and explore even more efficient scanning strategies.
