TLDR: This research introduces a novel Dual Actor Deep Deterministic Policy Gradient (DA-DDPG) algorithm for optimizing UAV-mounted Simultaneous Transmit and Reflect Reconfigurable Intelligent Surfaces (Aerial-STAR) in multi-user downlink communications. It addresses a more realistic coupled phase shift model for the STAR-RIS and jointly optimizes UAV trajectory, base station beamforming, and RIS coefficients. The study also proposes a Harmonic Fairness Index (HFI) to ensure equitable resource distribution. Results show DA-DDPG significantly outperforms other RL algorithms, 3D UAV trajectory is crucial for efficiency, HFI improves fairness, and there’s a trade-off between RIS size and UAV energy consumption.
The quest for next-generation wireless networks, beyond 5G and into 6G, demands significant advancements in spectrum efficiency, energy efficiency, ultra-low latency, and extensive coverage. Two promising technologies at the forefront of this evolution are Reconfigurable Intelligent Surfaces (RIS) and Unmanned Aerial Vehicles (UAVs).
Conventional RIS technology acts like a “smart mirror,” dynamically adjusting the phase of electromagnetic waves to enhance signal strength and mitigate interference. However, it’s limited to serving users only in its reflection space. This limitation is overcome by Simultaneous Transmit and Reflect Reconfigurable Intelligent Surfaces (STAR-RIS), which can both transmit and reflect signals, providing 360-degree coverage and greater flexibility.
While STAR-RIS offers significant advantages, its effectiveness can be hampered by fixed deployments, especially with mobile users or in dynamic environments. This is where UAV-mounted STAR-RIS, or Aerial-STAR, comes into play. By integrating STAR-RIS with UAVs, we gain the benefits of mobility and the ability to establish line-of-sight (LoS) communication, allowing for dynamic positioning and real-time adaptation to changing user distributions.
A recent study by Danish Rizvi and David Boyle from Imperial College London delves into a novel multi-user downlink communication system utilizing an Aerial-STAR. This research, detailed in their paper “Dual Actor DDPG for Airborne STAR-RIS Assisted Communications”, addresses a critical gap in existing research by moving beyond the common, often unrealistic, assumption of independent Transmission and Reflection Coefficients (TRC) in STAR-RIS. Instead, they explore a more practical coupled TRC phase shift model, where the transmit and reflect coefficients are interdependent.
Optimizing Performance with Dual Actor Deep Deterministic Policy Gradient (DA-DDPG)
The core challenge lies in jointly optimizing several complex factors: the UAV’s trajectory, the active beamforming vectors at the base station, and the passive RIS TRCs, all while considering the UAV’s energy constraints. To tackle this, the researchers propose a novel Dual Actor Deep Deterministic Policy Gradient (DA-DDPG) algorithm. This algorithm is unique because it employs two separate actor networks – one for high-dimensional continuous actions (like UAV trajectory, reflect phase, and beamforming) and another for discrete actions (like transmit phase selection). This specialized approach allows the system to effectively manage the complex hybrid action space that arises from practical STAR-RIS designs.
Ensuring Fairness with the Harmonic Fairness Index (HFI)
Beyond maximizing overall communication efficiency, the study also emphasizes communication fairness among users. To achieve this, the authors introduce a new fairness metric called the Harmonic Fairness Index (HFI). Unlike traditional metrics like Jain’s Fairness Index (JFI), HFI is more sensitive to disparities in user throughput, particularly for underserved users. By incorporating HFI into the reward function for the DA-DDPG algorithm, the system is incentivized to ensure a more equitable distribution of resources, aiming for a minimum data rate for all users.
Key Findings and Insights
The simulation results from this study offer compelling insights:
- The proposed DA-DDPG algorithm significantly outperforms conventional DDPG and DQN-based solutions, achieving 24% and 97% higher accumulated rewards, respectively. This highlights the effectiveness of its dual-actor architecture in handling complex hybrid action spaces.
- Optimizing the UAV’s trajectory in three dimensions (3D) leads to a 28% higher communication efficiency compared to optimizing only in two dimensions or just altitude. This underscores the importance of full spatial flexibility for Aerial-STAR systems.
- The HFI-based reward function proves highly effective, resulting in a 41% lower Quality of Service (QoS) denial rate compared to other benchmarks, demonstrating its ability to promote fairer communication.
- Mobile Aerial-STAR systems show superior performance over their fixed counterparts, and the coupled phase STAR-RIS model outperforms dual Transmit/Reflect RIS and conventional RIS setups.
- An important trade-off was identified: while increasing the number of RIS elements (size) can boost throughput, it also quadratically increases aerodynamic drag and energy demand on the UAV. This means that simply making the RIS larger doesn’t always improve communication efficiency, requiring careful design considerations.
Also Read:
- Optimizing Information Freshness and Handover in Space-Air-Ground Networks with AI
- Paving the Way for Sustainable 6G: A Deep Dive into Energy-Aware Network Design
Future Outlook
This research paves the way for more efficient and fair next-generation wireless networks. The findings emphasize the potential of Aerial-STAR systems and the robustness of the DA-DDPG approach. Looking ahead, the authors envision an expanded DA-DDPG agent integrated into Open RAN-compatible base station architectures, allowing for flexible and robust network optimization within an open, multi-vendor ecosystem, coordinating multiple Aerial-STAR systems effectively.
