TLDR: PTRN-HAR is a novel Online Continual Learning (OCL) approach for sensor-based Human Activity Recognition (HAR) that enables models to efficiently adapt to new activities and changes in activity patterns without forgetting previously learned information. It achieves this by pre-training a feature extractor with contrastive loss and then freezing it, while using a lightweight relation module network for classification. This design significantly reduces computational costs, memory usage, and the need for extensive labeled data, outperforming existing state-of-the-art methods in performance and resource efficiency.
Imagine a smart device that can recognize your daily activities, like walking, running, or even just sitting. This is the goal of Human Activity Recognition (HAR) using sensors, often found in smart homes, healthcare, and manufacturing. However, a major challenge arises after these devices are deployed: people change how they perform activities, or new activities emerge. Traditional machine learning models struggle to adapt to these changes without forgetting what they already learned, a problem known as ‘catastrophic forgetting’.
To address this, researchers have developed Online Continual Learning (OCL) methods. OCL allows models to learn new information incrementally from a continuous stream of data while preserving previously acquired knowledge. While OCL is well-suited for sensor-based HAR, existing approaches often demand significant computational power and a large amount of labeled data for every new change. Recently, methods that use pre-trained models (PTM-based OCL) have shown great promise in fields like computer vision, but applying them to HAR has been difficult due to the diverse nature of HAR datasets and the scarcity of labeled data in real-world scenarios.
A new research paper introduces PTRN-HAR, a groundbreaking approach that successfully applies PTM-based OCL to sensor-based HAR. This method is designed to be highly efficient in terms of both computational resources and the amount of labeled data required, all while maintaining high performance. The core of PTRN-HAR lies in two key innovations:
A Smart Feature Extractor
Unlike previous PTM-based OCL methods that rely on massive datasets for pre-training, PTRN-HAR pre-trains its ‘feature extractor’ using a technique called contrastive loss with only a limited amount of data. This feature extractor is responsible for identifying general patterns in the sensor data. Once pre-trained, this component is ‘frozen’ during the streaming stage, meaning it doesn’t need to be continuously updated. This significantly reduces the computational burden, making the system much more resource-efficient.
The Relation Module Network
PTRN-HAR replaces the conventional classification layer with a novel ‘relation module network’. This module acts as a smart comparator, learning to classify activities by measuring the similarity between new data features and a small set of stored ‘replay’ examples. This design not only helps the model interpret features from new activities or activities performed in different ways, even with a frozen feature extractor, but also dramatically improves data efficiency by requiring fewer labeled samples for effective continual learning.
How PTRN-HAR Works in Practice
The process involves two stages. In the ‘pre-deployment’ stage, the feature extractor learns general patterns from a set of initial activities. Then, in the ‘streaming’ stage, as new sensor data arrives (which might include new activities or variations of old ones), the frozen feature extractor processes it into compact ’embeddings’. The relation module network then uses these embeddings, along with a small, continuously updated set of ‘replay data’ (which are also embeddings, not raw sensor data), to classify the activities. This means only the lightweight relation module needs to be retrained periodically, rather than the entire complex model.
Also Read:
- Predicting High-Performing AI for Wearables Without Extensive Training
- PECL: Enhancing Human Activity Recognition with Multi-Domain Radar Sensing
Impressive Results and Efficiency
Experiments conducted on three public HAR datasets (PAMAP2, HAPT, and DSADS) demonstrate that PTRN-HAR significantly outperforms existing OCL approaches. It achieves a substantial improvement of 16.7–18.9% in Macro-F1 score, a metric particularly important for HAR applications where some activities might be less common than others. The method also shows remarkable stability as the number of new activities increases.
Crucially, PTRN-HAR proves to be highly resource-efficient. Tests on a Raspberry Pi 5, a common edge device, showed that its training time per new activity is significantly lower than other high-performing methods. It also requires much less memory because it stores compact data embeddings instead of bulky raw sensor data. Furthermore, PTRN-HAR is highly data-efficient, performing well even with as few as 10 labeled samples per new activity.
This research marks a significant step forward in making adaptive HAR systems practical for real-world deployment on resource-constrained devices, paving the way for smarter, more responsive wearable technology and intelligent environments.
