TLDR: This study compares deep learning (DL) and classical machine learning (ML) methods for classifying 24-hour movement behaviors (sleep, sedentary, light, and moderate-to-vigorous physical activity) using wrist-worn accelerometer data. It found that DL algorithms trained directly on raw acceleration signals showed slightly better accuracy (around 85%) than both DL and classical ML models trained on manually extracted features (70-81%). Random Forest, a classical ML algorithm, demonstrated strong performance, particularly in distinguishing light and moderate-to-vigorous physical activity, making it a competitive choice.
Understanding how people move throughout their day, including periods of sleep, sedentary behavior, light physical activity (LPA), and moderate-to-vigorous physical activity (MVPA), is crucial for health research. Traditionally, this has been a challenging task, often relying on subjective self-reported methods or complex manual feature extraction from wearable sensor data. However, with the rise of wrist-worn accelerometers, there’s a growing interest in using artificial intelligence to automatically classify these 24-hour movement patterns.
A recent study, titled “Classification of 24-hour movement behaviors from wrist-worn accelerometer data: from handcrafted features to deep learning techniques,” explored and compared two major artificial intelligence approaches: classical machine learning (ML) and deep learning (DL). The research, conducted by Alireza Sameh, Mehrdad Rostami, Mourad Oussalah, and Vahid Farrahi, aimed to determine which method offers better accuracy in predicting these daily movement intensities from raw accelerometer signals.
The Challenge of Movement Classification
Wearable accelerometers provide objective data, but translating this raw data into meaningful categories like sleep or different activity levels isn’t straightforward. Classical ML methods, such as Random Forest (RF), Support Vector Machine (SVM), and Artificial Neural Networks (ANN), have been widely used. However, they typically require ‘handcrafted features’ – specific statistical measurements extracted from the raw signals. This process is time-consuming, complex, and demands specialized knowledge.
Deep learning, on the other hand, offers the advantage of automatically learning and extracting these features directly from raw data, potentially simplifying the process and improving performance. Yet, its effectiveness in classifying 24-hour movement behaviors from raw accelerometer data remained largely unexplored.
Comparing the Approaches
The researchers utilized an open-access dataset from 151 adults who wore wrist-worn accelerometers for 24 hours. The raw acceleration signals were divided into 10-second segments. The study then followed two main experimental paths:
-
Deep Learning with Raw Signals: Four DL algorithms – Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM), Gated Recurrent Units (GRU), and One-Dimensional Convolutional Neural Network (1D-CNN) – were trained directly using the raw acceleration data.
-
Deep Learning and Classical ML with Handcrafted Features: A comprehensive set of 104 handcrafted features (from both time and frequency domains) was extracted from the 10-second signal windows. These features were then used to train the same four DL algorithms, as well as six classical ML algorithms: Random Forest (RF), Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), Logistic Regression (LR), Artificial Neural Network (ANN), and Decision Tree (DT).
The dataset was carefully split into training, validation, and test sets to ensure the models were evaluated on unseen data, reflecting real-world performance. The study also investigated whether selecting a smaller, more relevant subset of handcrafted features would impact the results.
Key Findings
The results provided valuable insights into the performance of these different AI techniques:
-
Deep Learning with Raw Signals Performed Best: LSTM, BiLSTM, and GRU algorithms, when trained directly on raw acceleration signals, achieved the highest overall accuracy of approximately 85%. The 1D-CNN had an accuracy of about 80%. These DL models were particularly good at classifying sleep and sedentary behaviors but showed more confusion between LPA and MVPA.
-
Handcrafted Features Reduced DL Performance: When the DL algorithms were trained using the handcrafted features instead of raw signals, their overall accuracy slightly decreased to around 79-80%. This suggests that the raw, richer data allows DL models to learn more effectively.
-
Classical ML was Competitive: Among the classical ML algorithms trained with handcrafted features, XGBoost achieved the highest accuracy at 81.1%, closely followed by Random Forest and ANN at 80%. Notably, Random Forest demonstrated less confusion in distinguishing between LPA and MVPA compared to other classical ML methods and even some DL methods.
-
Feature Selection Had Minimal Impact: Reducing the number of handcrafted features using a selection method had only a minor effect (1-4% variation) on the performance of both DL and classical ML algorithms.
Also Read:
- Personalized Sleep Forecasts from Wearables: An Adaptive AI Framework
- Enhancing Indoor Location Tracking in Homes with UWB and Deep Learning
Conclusion
The study concluded that deep learning methods, especially LSTM, BiLSTM, and GRU, when trained directly on raw accelerometer signals, offer a slight improvement in predicting 24-hour movement behaviors compared to approaches relying on handcrafted features. This highlights the potential of DL to automate feature extraction and leverage the full complexity of raw data.
However, the research also underscored the strong performance of classical ML algorithms, particularly Random Forest. RF proved to be a robust choice, showing less confusion in classifying light and moderate-to-vigorous physical activity, making it a reasonable option for predicting these movement intensities. The findings suggest that while DL with raw signals holds promise, classical ML with well-engineered features remains a highly competitive and effective approach.
For more detailed information, you can refer to the full research paper here.
