TLDR: Blexer v3 is a new AI-driven serious games ecosystem designed for personalized rehabilitation. It uses multimodal sensors to collect real-time physiological and emotional data from users. A central Context Awareness Module (CAM) interprets this data to infer user states like fatigue and engagement, then generates adaptive strategies. An Intelligent Play Module (IPM) translates these strategies into dynamic adjustments of game difficulty and content, including procedural content generation. This modular approach aims to improve therapeutic outcomes, enhance user engagement, and reduce clinician workload by providing a flexible and intelligent rehabilitation platform.
Serious games, which are video games designed for a primary purpose beyond pure entertainment, are increasingly being used in fields like education, military, and healthcare. In healthcare, these games show significant promise for rehabilitation, offering engaging experiences that can improve patient adherence and recovery. However, many existing rehabilitation game systems are often tied to specific games and lack the flexibility to adapt to individual patient needs across different scenarios.
A new research paper outlines the design vision for Blexer v3, an innovative, modular, and AI-driven rehabilitation ecosystem built around serious games. This system aims to overcome the limitations of previous versions and current approaches by integrating advanced multimodal sensing, real-time reasoning, and intelligent control to provide truly personalized interventions.
The Evolution of Blexer
The Blexer system has been under development since 2015 by the Research Group for Acoustics and Multimedia Applications (GAMMA) at Universidad Politécnica de Madrid (UPM). Its journey highlights the progression towards more adaptive and intelligent rehabilitation tools:
- Blexer v1: This initial version used Kinect for motion capture and allowed therapists to set fixed training parameters via a web platform. While it enabled remote supervision, it lacked real-time adaptability and couldn’t sense a player’s emotional or physiological states.
- Blexer v2: An improvement that integrated games within Unity and used Kinect Xbox One. Therapists could manually adjust difficulty settings based on performance trends visualized on the Blexer-med platform. However, adjustments were still human-driven, subjective, and couldn’t respond to real-time changes in a patient’s fatigue or emotional state.
These limitations paved the way for Blexer v3, which introduces a more advanced, autonomous, and context-aware architecture.
Blexer v3: A Smart, Self-Adaptive Ecosystem
Blexer v3 is designed to be a modular and intelligent platform that adapts in real-time to a patient’s condition. It integrates various sensing modalities and AI-driven modules for decision-making and feedback:
- Sensor Module (SM): This module collects real-time physiological and behavioral data from various sensors, including Kinect for motion, and wearable devices like the Polar H10 chest strap and Bangle.js 2 smartwatch for heart rate, RR intervals, and accelerometer data. This broadens the system’s understanding of the user’s physical state.
- Emotion Module (EM): Using facial expression analysis from the MoCap camera, this module detects a user’s emotional state in real-time. It simplifies emotions into four core states (positive, neutral, surprise, and negative) to improve reliability and support adaptive gameplay.
- Context Awareness Module (CAM): This is the central reasoning core. It fuses all the multimodal data (sensor data, emotions, game progress, past history, medical recommendations, preferences, time of day) to infer latent user states like cognitive workload, emotional engagement, and physical fatigue. CAM then generates high-level adaptive control directives, such as recommending a type of task or adjusting parameters like repetitions or duration.
- Intelligent Play Module (IPM): Acting as the universal actuator, IPM interprets CAM’s directives and translates them into concrete, in-game adjustments. This includes modifying task difficulty, switching exercises, and adjusting feedback intensity. A key feature is the integration of Procedural Content Generation (PCG), which allows for the dynamic creation and reordering of game content and levels to maintain engagement and variability.
- Interactive Assistance Module (IAM): This module supports therapists by continuously monitoring patient performance, providing real-time feedback, and alerting them to abnormalities like increased fatigue. It also offers visualizations of inferred user states and control strategies, which is particularly useful for remote rehabilitation.
A core innovation of Blexer v3 is the clear separation between the Context Awareness Module (CAM) and the Intelligent Play Module (IPM). CAM handles the high-level reasoning and strategy generation, while IPM is responsible for executing these strategies within the game. This decoupling ensures consistency in adaptation, flexibility across different games, and scalability, as one IPM can coordinate many exercises.
Also Read:
- Agentic-AI Healthcare: A New Framework for Secure, Multilingual, and Explainable Digital Health
- Unpacking Emotions in Speech: How AI Distinguishes Context from Personal Feelings
Personalized Rehabilitation and Future Outlook
Blexer v3 aims to provide personalized rehabilitation experiences by continuously adapting game difficulty (Dynamic Difficulty Adjustment – DDA) and content based on a user’s cognitive, emotional, and physiological states. This approach is expected to improve therapeutic outcomes and reduce the workload on clinicians by offering automated support and transparent feedback.
The researchers plan to conduct initial studies with a small group of players to test the system’s feasibility and identify areas for improvement. Future developments will explore more advanced reinforcement learning models to further enhance personalization. Blexer v3 represents a significant step towards creating emotionally responsive and data-driven therapeutic ecosystems. For more in-depth technical details, you can read the full research paper here: Design Process of a Self Adaptive Smart Serious Games Ecosystem.
