Reinforcement Learning Techniques for Next Generation Wireless Networks

  • Abdulmajeed Alenezi

Student thesis: Phd

Abstract

The number of mobile devices in indoor environments has dramatically increased, and the capacity of conventional RF wireless networks may not be enough to support the indoor traffic demand. Users' applications, such as texting, 4K video streaming, and virtual reality have substantial differences in terms of data rate requirements. A heterogeneous network is one of the most promising approaches to improve indoor coverage and throughput. Recently, visible light communication (VLC) systems have emerged as a complementary unlicensed media. In this thesis, we proposed a hybrid WiFi-VLC system wherein multiple VLC access points (APs) coexist with a WiFi AP. A number of indoor users can share the hybrid WiFi-VLC system. All users employ WiFi for the uplink, and one access point (WiFi or VLC) is assigned to each user. We presented reinforcement learning algorithms that can be implemented at the WiFi AP to aid in the selection of an access point for each user. Moreover, we proposed a new federated Q-learning (FQL) algorithm, in which each VLC AP performs local Q-learning and updates the global model at the WiFi AP. Knowledge transfer using a neural network (NN) was proposed to further reduce the FQL's convergence speed. We evaluated the performance of the proposed approaches using different objective functions such as sum-rate and max-min. Finally, we proposed a global Q-learning approach for a macro base station to solve the resource allocation problem in a dense femtocell network. The reward function was designed to maintain the quality of service (QoS) for a macro user and maximize the sum capacity of the femtocell users. Numerical simulations showed that the derived Q-learning algorithms in this thesis improved the network performance.
Date of Award31 Dec 2022
Original languageEnglish
Awarding Institution
  • The University of Manchester
SupervisorHujun Yin (Supervisor) & Khairi Hamdi (Supervisor)

Keywords

  • Visible Light Communication
  • Resource Allocation
  • Reinforcement Learning
  • Hybrid WiFi-VLC

Cite this

'