TLDR: CARE is a new framework for recognizing daily activities (ADLs) in smart homes using event-triggered sensors. It combines sequence-based and image-based sensor data representations through a novel contrastive alignment technique (SICA). This approach allows the model to leverage both temporal and spatial information effectively, leading to state-of-the-art performance and improved robustness against sensor issues and layout changes, all within an efficient, end-to-end system.
As our global population ages, supporting older adults in maintaining their independence for as long as possible becomes increasingly vital. A key aspect of independent living is the ability to perform Activities of Daily Living (ADLs) such as cooking, dressing, and personal hygiene. Ambient Assisted Living (AAL) technologies, particularly those using unobtrusive ambient sensors embedded in homes, offer a promising solution for monitoring these activities without invading privacy or requiring wearables.
However, recognizing ADLs from these event-triggered ambient sensors presents significant challenges. Unlike continuous data from wearable sensors, ambient sensors generate sparse, irregular, and often noisy data. For instance, a motion sensor only activates when someone passes by, and a door sensor only when it’s opened or closed. Furthermore, individuals may perform the same ADL with variations in order, speed, or completion, making accurate recognition difficult.
Existing methods for encoding this sensor data typically fall into two categories: sequence-based and image-based. Sequence-based approaches excel at capturing the temporal order of events but can be sensitive to noise and often lack spatial awareness. Image-based methods, on the other hand, transform sensor events into visual patterns, leveraging global spatial correlations. However, they can distort true sensor layouts and compress fine-grained temporal details. Simple combinations of these methods, like merely concatenating features, often fail to truly align the complementary strengths of both views.
To address these limitations, researchers have introduced CARE (Contrastive Alignment for ADL Recognition from Event-Triggered Sensor Streams), an innovative end-to-end framework. CARE aims to unify sequence- and image-based representations by jointly optimizing representation learning through a novel technique called Sequence–Image Contrastive Alignment (SICA) and classification via cross-entropy. This ensures that the model not only aligns information across different data views but also learns to distinguish between various activities effectively.
The CARE framework integrates several key components:
Robust Temporal Encoding
CARE employs a time-aware and noise-resilient sequence encoding method. It uses ‘temporal binning’ to assign a coarse-grained temporal embedding (e.g., hourly bins) to each sensor event, capturing daily context without being overly sensitive to minute-level fluctuations. Additionally, ‘frequency-based event filtering’ is used to remove unreliable sensor activations that occur only once or a few times within an activity, thereby reducing noise and shortening sequences while preserving important information.
Spatially-Informed and Frequency-Sensitive Image Representation
Instead of a single image, CARE generates a dual-perspective image representation. A ‘temporal image’ captures event order, sensor identity, and signal state, highlighting temporal dynamics. A ‘spatial image’ leverages the actual floorplan coordinates of sensors, representing sensor positions as nodes and transitions between consecutive events as edges. Node colors indicate activation frequency, and edge darkness shows transition frequency. These two images are then combined to form a composite image, providing a rich, spatially-informed, and frequency-sensitive representation.
Sequence–Image Contrastive Alignment (SICA)
This is the core of CARE. SICA introduces a supervised contrastive mechanism that explicitly aligns embeddings from both the sequence and image views into a shared latent space. In simple terms, it pulls together representations of the same activity (whether from sequence or image data, or even different instances of the same activity) while pushing apart representations of different activities. This process enforces both consistency across different data views and clear separation between different activity classes, leading to more robust and discriminative representations.
Also Read:
- Optimizing Vision-Language-Action Models for Robotics: A Deep Dive into Efficiency
- Dynamic Learning: Enhancing Acoustic Scene Classification for Unseen Devices
Unified End-to-End Framework
Unlike traditional two-stage approaches where representation learning and classification are separate, CARE integrates both into a single optimization stage. This means the model learns aligned representations and makes activity predictions simultaneously, making it more efficient and ready for real-world deployment.
Evaluated on three CASAS datasets (Milan, Cairo, and Kyoto7), CARE achieved state-of-the-art performance, with F1-scores of 89.8% on Milan, 88.9% on Cairo, and 73.3% on Kyoto7. The framework demonstrated superior performance compared to traditional machine learning, sequence-based, image-based, graph-based, and even recent language-inspired models. Ablation studies confirmed the critical role of cross-view alignment, showing significant gains over uni-view or within-view alignment methods, and outperforming naive feature concatenation.
Furthermore, CARE proved robust to real-world challenges such as sensor malfunctions and layout variability, maintaining high accuracy even when sensor events were corrupted or positions perturbed. This resilience is attributed to its noise-filtering preprocessing, the redundancy provided by its dual-branch architecture, and the strong regularization effect of the SICA loss.
In conclusion, CARE represents a significant advancement in ADL recognition from event-triggered sensor streams. By explicitly aligning temporal and spatial cues through supervised contrastive learning, it not only boosts performance but also enhances robustness, paving the way for more reliable and scalable ambient intelligence systems in smart homes. For more details, you can refer to the full research paper.
