TLDR: This research proposes a Human-Machine Coordinated Dynamic Bandwidth Allocation (HMC-DBA) scheme for Extended Reality (XR) in human-to-machine (H2M) collaborations over future enterprise networks. By using AI models like BiLSTM to predict human head movements in advance, the system can pre-orient machine cameras and proactively allocate network bandwidth. This significantly reduces end-to-end latency and jitter, improves the immersive experience, and optimizes network resource utilization, especially for high-quality XR content.
The world of technology is rapidly advancing, with innovations like Extended Reality (XR) and human-to-machine (H2M) collaborations becoming central to industrial and social progress. However, ensuring a smooth, immersive experience in these scenarios, especially when humans and machines interact across vast distances, presents a significant challenge: synchronizing XR content with human head movements in real-time to prevent issues like cyber-sickness.
A new research paper, titled “User Head Movement-Predictive XR in Immersive H2M Collaborations over Future Enterprise Networks,” by Sourav Mondal and Elaine Wong, addresses this critical problem. This work introduces a novel approach to overcome the inherent time lag between a human’s head movements and a remote machine’s camera, which is crucial for maintaining an ideal immersive experience.
The Core Challenge: Lag and Bandwidth
In H2M collaborations, a human’s head-mounted device (HMD) sends sensing information to an application server, which then translates it into commands for a machine’s camera. The machine orients its camera, and real-time video frames are sent back to the human’s HMD. The problem arises because if there’s a consistent delay of 20 milliseconds or more between the human’s head movement and the visual scene, the brain detects a de-synchronization, leading to cyber-sickness. Furthermore, high-quality XR video demands not only high data rates but also extremely low latency and jitter between consecutive frames. For instance, 4K video at 60 frames per second requires about 90 Mbps, with a latency budget of 10-30 milliseconds.
Adding to the complexity, the size of XR frames can vary significantly based on how much the human’s head moves. Sudden, rapid head movements lead to larger angular shifts between frames, reducing inter-frame correlation and thus increasing frame sizes. This, in turn, abruptly increases the demand for network bandwidth. If this bandwidth isn’t immediately available, packets can be delayed, degrading the quality of experience.
A Predictive Solution: HMC-DBA
To tackle these issues, the researchers propose a Human-Machine Coordinated Dynamic Bandwidth Allocation (HMC-DBA) scheme. The core innovation lies in predicting the human’s future head movements with high accuracy. By knowing where the human’s head will be, the machine’s camera can be pre-oriented in advance. This proactive adjustment means the camera starts rotating steadily before the human’s actual movement, allowing the resource demand to increase gradually over a longer period. This significantly reduces the probability of delayed packet transmission and helps maintain a satisfactory quality of experience.
The scheme leverages advanced artificial intelligence models, particularly Bidirectional Long Short-Term Memory (BiLSTM) networks, for predicting human head orientations. BiLSTM networks are highly effective for time-series data like head movements because they process input sequences in both forward and backward directions, capturing subtle temporal dependencies and improving prediction accuracy even for sudden movements.
How It Works in the Network
The HMC-DBA scheme is designed for deployment over Fiber-To-The-Room-Business (FTTR-Business) networks, which are high-capacity fiber networks suitable for industrial applications. In this setup, various network units (SFUs, MFUs, OLT) work in coordination:
- When data (human head movements, XR frames, or background traffic) arrives at the Subordinate FTTR Units (SFUs), they send bandwidth requests.
- Main FTTR Units (MFUs) then grant bandwidth not just based on the immediate request, but on predicted future needs, reducing queuing time.
- A central Optical Line Terminal (OLT) with an Edge-AI server is the brain of the operation. It receives human head movement data, uses the AI module to predict future movements, and then instructs the machine’s camera to pre-orient.
- Crucially, the AI module also predicts future XR bandwidth requirements based on these predicted head movements and allocates network resources proactively.
This proactive allocation, combined with the pseudo-periodic nature of XR and HMD traffic, allows the system to pre-allocate resources, further reducing transmission latency and jitter.
Also Read:
- Gaze-Guided Robots: Enhancing Efficiency and Robustness with Human-Inspired Vision
- Detecting Robot Errors Through Human Signals: The ERR@HRI 2.0 Challenge
Impressive Results
Through extensive simulations, the researchers demonstrated the effectiveness of their HMC-DBA scheme. The BiLSTM method achieved a normalized Root Mean Square Error (RMSE) of less than 0.1 for predicting all components of human head movements (yaw, pitch, and roll), significantly outperforming other prediction methods like persistence, moving average, and ARIMA.
The most compelling result is the dramatic reduction in end-to-end latency. While state-of-the-art baseline bandwidth allocation schemes struggled to meet the ideal quality of experience requirements for high-quality XR (e.g., 8K frames), the proposed HMC-DBA successfully ensured these stringent requirements were met. This is largely due to the reduced bandwidth requests for transmitting XR frames, as the predictive mechanism allows for smaller frame sizes during rapid head movements.
This research marks a significant step forward in enabling truly immersive and seamless human-to-machine collaborations over future enterprise networks. For more in-depth information, you can read the full paper available at arXiv.org.
