DeepWiN: Deep Graph Reinforcement Learning for User-Centric Radio Access Networks Automation

Abstract

The future cellular networks are expected to support an increasing number of users with heterogeneous applications, requiring varying network resources. Therefore, the 6G and beyond cellular networks need to be elastic, and user-centric. User-centric Radio Access Networks (UCRAN), with virtual cells (S-zones), can provide on-demand connectivity, coverage and quality of service to different user applications while optimizing the network for energy efficiency, area spectral efficiency, reliability and user service rate. However, with high variability in the network, due to user mobility and fading, the selection of S-zone sizes which optimize the network performance for multiple types of users simultaneously becomes a challenge. Therefore, to automate the selection of S-zone sizes dynamically, we propose deep graph reinforcement learning (DGRL), a Soft actor-critic model integrated with Graph neural network. DGRL infers from DeepWiN, a graphical representation of UCRAN that encodes the non-euclidean topology of the network along with its euclidean features, effectively encapsulating the wireless domain knowledge of the network configuration. Our experiments show that the deep graph reinforcement learning can learn to optimize S-zone sizes with 15% fewer training episodes in comparison to the legacy neural-network-based reinforcement learning, hence demonstrating the advantage of network topology-awareness for artificial intelligence.