TLDR: CognitiveArm is a new EEG-controlled prosthetic arm system that uses advanced deep learning models optimized for real-time operation on small, embedded AI hardware. It achieves high accuracy (up to 90%) in classifying brain signals for movement, integrates voice commands for seamless control, and overcomes limitations of existing systems by ensuring low latency and high efficiency through techniques like model compression.
Imagine controlling a prosthetic arm simply by thinking about moving it. This is no longer science fiction, thanks to a groundbreaking system called CognitiveArm. Developed by researchers at NYU Abu Dhabi, CognitiveArm is an innovative prosthetic arm system that uses brain signals, specifically electroencephalography (EEG), to enable real-time, precise control, even on small, power-efficient devices.
The Challenge of Prosthetic Control
Traditional prosthetic limbs often rely on residual muscle activity, which can be unreliable or absent for individuals with conditions like ALS or spinal cord injuries. EEG-based brain-computer interfaces (BCIs) offer a promising alternative by directly interpreting brain signals. However, existing EEG-controlled systems often face significant hurdles. Many require powerful computers or cloud servers for processing, leading to delays (latency) that make real-time control difficult. Deep learning models, while powerful, are computationally intensive, making them hard to run on small, embedded devices with limited resources. Furthermore, ensuring accuracy while keeping the system fast and efficient is a major challenge.
Introducing CognitiveArm: A Smart Solution
CognitiveArm tackles these challenges head-on. It’s designed to run entirely on embedded AI hardware, ensuring low latency and real-time responsiveness without sacrificing accuracy. The system integrates several key components:
-
EEG Data Acquisition: It uses an OpenBCI UltraCortex Mark IV EEG headset to capture brain signals from 16 electrodes. An open-source library called BrainFlow facilitates high-fidelity data acquisition and streaming.
-
Smart Signal Processing: Raw EEG signals are pre-filtered to remove noise and interference, ensuring clean data for analysis. This includes removing electrical noise and artifacts like eye blinks.
-
Custom Dataset Generation: The researchers collected a unique EEG dataset from participants imagining specific actions: moving their right hand, moving their left hand, or staying idle. This data was meticulously labeled to train the system.
-
Optimized Deep Learning: CognitiveArm employs advanced deep learning models like Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Transformers. To find the best balance between accuracy and efficiency, the system uses an ‘evolutionary search’ algorithm, which intelligently explores different model configurations and hyperparameters.
-
Model Compression: To make these complex models suitable for embedded devices, techniques like ‘pruning’ (removing unnecessary connections in the neural network) and ‘quantization’ (converting data to lower precision) are applied. This significantly reduces memory usage and speeds up inference without major accuracy loss.
-
Voice Command Integration: For enhanced user interaction, CognitiveArm also supports voice commands. Users can seamlessly switch modes (e.g., control the elbow, arm, or fingers) using simple voice prompts, enabling multi-action control for everyday tasks like a handshake or picking up a cup.
Also Read:
- Guiding Hands: AI and Muscle Stimulation for Enhanced Skill Learning
- Advancing Emotion Recognition: A Robust EEG Feature Selection Approach for Incomplete Data
Real-World Performance
The full-scale prototype of CognitiveArm, interfaced with the OpenBCI headset, demonstrated impressive results. It achieved up to 90% accuracy in classifying the three core actions (left, right, idle). The optimized models, particularly a combination of CNN and Transformer, achieved this high accuracy with very low inference times, making real-time control feasible. For instance, a 70% pruned model maintained over 90% accuracy while significantly reducing processing time, proving its practicality for edge devices like the NVIDIA Jetson Orin Nano, where the system is deployed.
This research marks a significant step forward in making brain-controlled prosthetics more accessible, efficient, and practical for daily use. By integrating advanced deep learning with embedded hardware and voice commands, CognitiveArm offers a robust and responsive solution for individuals seeking enhanced control over their prosthetic limbs. You can read more about this research in the paper: CognitiveArm: Enabling Real-Time EEG-Controlled Prosthetic Arm Using Embodied Machine Learning.
