TLDR: PECL (Parallel-EfficientNet-CBAM-LSTM) is a new deep learning network designed for radar-based human activity recognition. It addresses limitations of previous methods by simultaneously processing radar data from three complementary domains (Range-Time, Doppler-Time, Range-Doppler). The network incorporates a Convolutional Block Attention Module (CBAM) for better spatial and channel feature extraction and Long Short-Term Memory (LSTM) units in temporal branches to capture dynamic movement sequences. PECL achieves 96.16% accuracy, significantly outperforming existing methods, especially in distinguishing similar actions like ‘pick up’ and ‘drink,’ while maintaining moderate computational complexity.
Human Activity Recognition (HAR) is becoming increasingly vital, especially with an aging global population. From monitoring elderly individuals in their homes to assisting in hospitals, HAR technology offers significant benefits. Traditionally, HAR systems have relied on contact-based sensors like accelerometers or non-contact visual cameras. However, these methods come with drawbacks: contact sensors can be uncomfortable for prolonged use, and cameras raise significant privacy concerns, are affected by lighting, and can be obscured.
This is where radar systems step in. Millimeter-wave radar offers a non-intrusive, privacy-preserving solution that works regardless of lighting conditions or steam, making it ideal for sensitive areas like bathrooms where falls often occur. Despite these advantages, existing radar-based HAR research often falls short by focusing on single-domain radar signals and overlooking the crucial temporal aspects of human movement. This makes it difficult to accurately distinguish between similar actions, leading to misclassifications.
Introducing PECL: A New Approach to Radar-Based HAR
To overcome these limitations, researchers have developed a novel network called Parallel-EfficientNet-CBAM-LSTM, or PECL. This innovative system is designed to process radar data from three complementary domains simultaneously: Range-Time, Doppler-Time, and Range-Doppler. By combining information from these diverse perspectives, PECL aims to capture a more complete picture of human activities, improving both accuracy and robustness.
How PECL Works
The PECL network operates with a parallel architecture, meaning it processes different types of radar data streams at the same time. Each stream, or ‘branch,’ is tailored to extract specific information:
-
Range-Time Domain: This branch tracks how a target’s distance changes over time, revealing movement patterns.
-
Doppler-Time Domain: This branch focuses on velocity variations over time, which is excellent for identifying periodic movements like walking or falling.
-
Range-Doppler Domain: This 2D representation captures motion patterns across both distance and velocity, helping to distinguish between stationary and dynamic activities.
At its core, PECL utilizes a modified EfficientNet-B0 architecture, known for its balance between performance and computational efficiency. A key enhancement is the replacement of the standard Squeeze-and-Excitation (SE) module with a Convolutional Block Attention Module (CBAM). While SE modules primarily focus on ‘what’ features are important across channels, CBAM goes a step further by also considering ‘where’ in the spatial dimension those important features are located. This dual attention mechanism helps the network pinpoint critical information within the radar spectrograms.
Furthermore, PECL incorporates Long Short-Term Memory (LSTM) modules into its Range-Time and Doppler-Time branches. LSTMs are particularly adept at understanding sequences and temporal dependencies, which is crucial for distinguishing actions that might look similar in a snapshot but differ in their timing, such as ‘picking up an object’ versus ‘drinking from a cup.’ The Range-Doppler branch, which doesn’t inherently contain temporal sequences in the same way, uses a linear layer followed by max pooling to efficiently extract salient features.
The outputs from these three specialized branches are then combined, or ‘fused,’ to create a comprehensive feature representation. This fused information is then fed into a final classification layer to identify the human activity.
Impressive Results and Efficiency
Experimental results demonstrate PECL’s superior performance. It achieved an accuracy of 96.16% on a widely used radar dataset, outperforming existing methods by at least 4.78%. Notably, PECL excelled at distinguishing between easily confused actions like ‘Pick Up’ and ‘Drink,’ achieving 89.77% and 90.38% accuracy for these categories, respectively. This is a significant improvement over previous models.
Despite its strong performance, PECL maintains a moderate model complexity, with 23.42 million parameters and 1324.82 million FLOPs. This parameter-efficient design ensures that the network is not overly computationally expensive, making it suitable for practical applications.
The research also confirmed PECL’s generalization ability on external datasets, showing that its individual components are effective and that multi-domain fusion further enhances accuracy. For more technical details, you can refer to the full research paper here.
Also Read:
- A Novel System for Real-time Human Motion and Gesture Recognition
- Adapting Fault Prediction in Smart Grids with Prototype-based Continual Learning
Looking Ahead
While PECL represents a significant leap forward in radar-based HAR, the researchers are already planning future enhancements. These include conducting more extensive statistical analyses, expanding the dataset to include more diverse and granular action categories (e.g., different types of falls), optimizing the model for deployment on resource-constrained devices, and exploring multi-modal sensing by integrating radar data with other inputs like RGB imagery for even greater robustness.
