TLDR: DEPF is a new UAV multispectral object detector designed to overcome challenges in low-light conditions and small target detection. It uses a Dual-Domain Enhancement Module (DDE) to brighten images and recover details, and a Priority-Guided Mamba Fusion Module (PGMF) to intelligently fuse information by prioritizing local targets. Experiments show DEPF outperforms existing methods in accuracy and computational efficiency on DroneVehicle and VEDAI datasets.
Multispectral object detection using Unmanned Aerial Vehicles (UAVs) is crucial for many applications, from monitoring to relief work. However, this technology faces significant hurdles. Low-light conditions in remote sensing images can reduce the effectiveness of combining information from different sources (multi-modality fusion). Additionally, when trying to identify small targets, redundant information during the fusion process can interfere with accurate modeling. Traditional transformer-based methods, while powerful, often come with high computational costs, making them difficult to deploy on resource-constrained UAV platforms.
To tackle these challenges, researchers Shucong Li, Zhenyu Liu, Zijie Hong, Zhiheng Zhou, and Xianghai Cao have proposed a novel solution called DEPF: a UAV multispectral object detector with dual-domain enhancement and priority-guided Mamba fusion. This new approach is inspired by the Mamba architecture, known for its linear computational complexity, which makes it more suitable for UAV applications. You can read the full research paper for more technical details here: DEPF Research Paper.
Dual-Domain Enhancement Module (DDE)
One of DEPF’s core components is the Dual-Domain Enhancement Module (DDE), designed specifically to improve low-light remote sensing images. The DDE module comprises two key blocks:
Cross-Scale Wavelet Mamba (CSWM)
The CSWM block focuses on enhancing the global brightness of images. It achieves this by applying a cross-scale Mamba scanning technique to the low-frequency components of the image, which contain most of the image’s overall information. This helps to brighten the darker areas effectively.
Fourier Details Recovery (FDR)
Even after brightness enhancement, images might still lack fine texture details. The FDR block addresses this by constructing a spectrum recovery network. This network works in the frequency domain to enhance the frequency spectra features, thereby recovering crucial texture and detail information that might be lost in low-light conditions.
Priority-Guided Mamba Fusion Module (PGMF)
The second crucial module in DEPF is the Priority-Guided Mamba Fusion Module (PGMF). This module is designed to improve the modeling of local small targets and minimize the interference from redundant information during the multi-modality fusion stage.
PGMF introduces an innovative concept called “priority scanning.” Instead of scanning images in a conventional Z-order, which often focuses on global context and can be distracted by background noise, PGMF assigns priority scores to different parts of the image based on modality differences. This allows the system to prioritize scanning from local target features, effectively reducing the impact of irrelevant background information while still retaining the overall global features. This is a significant improvement over previous Mamba-based fusion methods.
Experimental Results and Performance
The DEPF system was rigorously evaluated on two prominent datasets: DroneVehicle and VEDAI. The experimental results demonstrate that DEPF performs exceptionally well in object detection compared to state-of-the-art methods.
On the DroneVehicle dataset, DEPF significantly outperformed single-modality RGB methods and many multi-modality detectors, showing substantial improvements in mean Average Precision (mAP) scores. For instance, it surpassed RoITransformer (RGB) by 17.3 points in [email protected] and C2Former (RGB+Infrared) by 4.7 points in [email protected]. It also showed robust detection and classification in low-light scenarios, correctly identifying objects where other methods made errors.
Similarly, on the VEDAI dataset, DEPF achieved strong performance, outperforming many detectors with an [email protected] of 71.1%. While some methods showed higher scores for specific categories like “Car” or “Van,” DEPF demonstrated superior performance across other categories such as “Truck,” “Boat,” and “Other.”
Computational Efficiency
One of the key motivations behind DEPF was to achieve lightweight computation suitable for UAV platforms. The research paper shows that DEPF is indeed computationally efficient. It boasts a lower parameter count (80.3 MB) and GFLOPs (108.3 G) compared to several state-of-the-art methods like C2Former, reducing parameters by approximately 32.2%. Although its inference speed (FPS) is slightly lower than some competitors due to the computational costs of wavelet transforms and Fast Fourier Transforms in the DDE module, it still maintains a respectable 16 images/second.
Also Read:
- DepthVision: Enabling Robots to See Clearly in Challenging Conditions with LiDAR-Enhanced Vision
- Unlocking YOLOv10s Speed on Standard Hardware
Conclusion
DEPF represents a significant advancement in UAV multispectral object detection. By integrating a Dual-Domain Enhancement Module for improving low-light images and a Priority-Guided Mamba Fusion Module for intelligent, target-focused fusion, it effectively addresses critical challenges in the field. The innovative priority scanning mechanism in PGMF is a pioneering step in improving Mamba for multi-modality fusion. The strong experimental results confirm DEPF’s effectiveness, robustness, and computational efficiency, paving the way for more reliable drone-based remote sensing applications.
