TLDR: This research compares signal-based (sEMG) and video-based approaches for recognizing human motion intentions in human-robot collaboration (HRC) for construction. The study found that video-based models achieved higher accuracy (94%) but were slower (0.15s prediction time), while sEMG-based models were faster (0.04s) but less accurate (87%). The paper highlights the trade-offs between speed and accuracy, suggesting that the optimal choice depends on the specific application in construction HRC.
The construction industry is increasingly looking towards human-robot collaboration (HRC) to boost safety and efficiency. A crucial aspect of successful HRC is the robot’s ability to accurately and quickly understand human motion intentions. This research paper delves into a significant gap in current studies: a direct comparison of different data types—specifically signals and videos—for recognizing these human motion intentions.
The study, conducted by Charan Gajjala Chenchu, Kinam Kim, Gao Lu, and Zia Ud Din, explores how deep learning models can assess two distinct modalities to recognize workers’ early-stage motion intentions during drywall installation tasks. This comparison is vital for determining the most effective approach for HRC in real-world construction environments.
Signal-Based Approach: Speed and Portability
One approach utilized surface electromyography (sEMG) data, which captures muscle activity. A Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) model was employed for this signal-based method. This model achieved an accuracy of approximately 87% and was remarkably fast, making predictions on a sample input in about 0.04 seconds. The advantage of sEMG signals lies in their consistency, even in varying environmental conditions like lighting or obstructions, and their portability as sensors are attached directly to the skin. However, sEMG signals can be prone to noise and variability across different individuals, and wearing sensors might be uncomfortable for workers.
Video-Based Approach: Higher Accuracy, Broader Context
The second approach involved video sequences, processed using a pre-trained Video Swin Transformer model combined with transfer learning. This video-based method achieved a higher accuracy of 94%. However, it came with a trade-off: a longer average prediction time of 0.15 seconds for a similar input. Video-based systems offer the benefit of capturing the full visual context of surroundings, which is useful for operations involving multiple workers and complex behaviors. On the other hand, they demand significant computational power, are susceptible to issues like occlusions, varying viewpoints, and background clutter, and can be affected by lighting conditions.
Experimental Setup and Key Findings
The researchers designed an experiment involving three subjects performing four drywall installation tasks, simultaneously collecting sEMG signals and video data. The data was divided into “Intention” (the first second of an activity) and “Actual” groups, creating eight distinct target classes. Both models were trained and validated, with the video-based model consistently showing higher accuracy curves.
While the video-based model demonstrated superior classification accuracy (94.05% compared to 86.82% for the signal-based model), the signal-based model was significantly faster, about 3.7 times quicker in making predictions. This speed difference highlights a crucial trade-off: accuracy versus latency. For applications where immediate response is critical, such as safety hazard monitoring, the faster signal-based approach might be preferred. For tasks requiring a broader understanding of the environment and complex interactions, the higher accuracy of video-based methods could be more beneficial.
Also Read:
- EHWGesture: A Comprehensive Resource for Understanding Clinical Hand Movements
- Robots Learn to Grasp Like Humans Using Advanced Sensorimotor Integration
Conclusion and Future Directions
The study concludes that both sEMG and video modalities have unique strengths and weaknesses, suggesting that there isn’t a single universal approach for motion intention recognition in HRC. Future work could involve larger subject pools, more diverse construction tasks, and the development of fusion models that combine the strengths of both data types. This research provides valuable insights for systematically deploying these technologies to enhance human-robot collaboration in construction projects. You can read the full research paper here.
