TLDR: BaroPoser is a novel method that significantly improves real-time human motion tracking by combining data from Inertial Measurement Units (IMUs) and barometers found in everyday devices like smartphones and smartwatches. Unlike previous methods limited to flat surfaces, BaroPoser accurately estimates human pose and global translation, even on uneven terrain, by leveraging barometric readings for height changes and using a novel thigh-rooted coordinate system. It outperforms existing IMU-only approaches and has potential for AR/VR and fitness tracking.
Tracking human motion in 3D, a field known as motion capture (MoCap), is crucial for applications ranging from movie production and gaming to augmented reality (AR) and virtual reality (VR). While camera-based systems exist, they often struggle with visibility, occlusions, and lighting. Inertial Measurement Units (IMUs), small sensors that measure acceleration and rotation, offer a portable alternative. However, traditional IMU-based systems often require many specialized sensors, making them cumbersome and expensive.
Recent advancements have focused on using IMUs already present in everyday devices like smartphones and smartwatches. While this makes motion capture more accessible, these methods face challenges due to the sparse nature of sensor measurements and a lack of data for movements on uneven terrain. This often limits their accuracy for pose estimation and restricts them to tracking movements only on flat surfaces.
To address these limitations, researchers have introduced BaroPoser, a groundbreaking method that combines IMU data with barometric readings from a smartphone and a smartwatch to estimate human pose and global translation in real time. This is the first approach of its kind to leverage barometric information for full-body motion tracking from everyday devices.
How BaroPoser Works
BaroPoser utilizes a smartwatch worn on one wrist and a smartphone placed in the thigh pocket of the opposite side. The key innovation is the incorporation of barometric readings, which provide information about absolute altitude. These height changes offer valuable cues that significantly improve the accuracy of both human pose estimation and global translation, especially on non-flat terrain. This vertical awareness is particularly beneficial for activities involving altitude changes, such as stair climbing, squats, and jumps.
The system processes IMU data (accelerations and rotations) and barometric pressure readings. To ensure stable and accurate height estimation, a Kalman filter is used to fuse the barometric data with IMU readings, mitigating environmental noise. BaroPoser then divides the task into two main subtasks: height-aware local pose estimation and hybrid global translation estimation.
For local pose estimation, BaroPoser introduces a novel local thigh coordinate frame. By treating the thigh sensor as the root, both the input and output of the pose estimation network are represented in this local frame. This helps to disentangle local body poses from global orientation, allowing the deep learning model to focus on learning pure local motion signals. The relative height difference between the two sensors, derived from barometric data, is also fed into the network, further reducing ambiguity in pose estimation.
For global translation, BaroPoser decomposes the motion into horizontal and vertical components. The horizontal velocity is estimated by a neural network using sensor data. The vertical translation, however, is derived from the filtered height at the thigh, corrected by local thigh variations computed from the estimated pose. This hybrid approach enables the system to accurately track movements on non-flat terrain without requiring additional training data for such scenarios.
Also Read:
- PrivCLIP: A New Approach to User-Controlled Privacy in Sensor Data
- Unlocking Forest Secrets: Advanced 3D Segmentation with ForestFormer3D
Performance and Applications
BaroPoser has been rigorously evaluated on both public benchmark datasets and newly collected real-world recordings. Quantitative and qualitative results demonstrate that it consistently outperforms state-of-the-art methods that rely solely on IMUs with the same hardware configuration. This includes significant improvements in pose estimation metrics like SIP error, angular error, positional error, and mesh error, as well as enhanced global translation accuracy.
The ability to accurately track human motion in real time using common devices like smartphones and smartwatches opens up numerous possibilities. It can enhance AR/VR experiences by providing more precise body tracking, improve fitness tracking by accurately monitoring activities like stair climbing, and make motion capture technology more accessible for everyday users and casual scenarios.
While BaroPoser marks a significant leap forward, the researchers acknowledge certain limitations. The current method assumes a mean body shape and fixed sensor placements, which could be expanded for greater generalizability. Barometric sensors can also be susceptible to environmental factors, potentially leading to drift, an area for future research into more robust sensor fusion techniques. Additionally, the availability of real-world datasets combining IMU, barometer, and ground-truth motion data for uneven terrain remains a challenge.
Despite these limitations, BaroPoser showcases the immense potential of integrating barometric information to expand the applicability of inertial motion capture in unconstrained, real-world settings. For more technical details, you can refer to the full research paper: BaroPoser: Real-time Human Motion Tracking from IMUs and Barometers in Everyday Devices.
