With the great development of big data and increasing attention on privacy protection, distributed machine learning has received significant research interests in the last decade for its great ability in large-scale and privacy-related machine learning problems. Compared to traditional machine learning, distributed machine learning allows all participants to train a combined model, while keeping their private data locally stored. This thesis proposes a consensus-based distributed machine learning framework based on a decentralized communication topology, which frees the central master and exhibits great robustness and expansibility. The main contribution includes distributed supervised learning and distributed reinforcement learning. First, a distributed training method based on the consensus algorithm is proposed for neural networks connected over a decentralized topology, which only requires a single consensus step after every training step. It is proved that the distributed training allows all the agents over a decentralized topology to converge to the optimal model based on the convergence analysis on empirical risk and model parameter. Second, the distributed training method is promoted based on the heuristic adaptive consensus algorithm and stochastic variance reduced gradient for agents connected in switching communication topologies. Theoretical analysis shows that all agents in switching graphs can still converge to the optimum and the stochastic variance reduced gradient reduces the variance introduced by stochastic gradient with only a little extra computational cost. Third, the error-compensated compression method with bit-clipping is applied in distributed training to compress the model parameter before sharing, which significantly saves communication costs with little decrease in model accuracy and is suitable for both IID and non-IID datasets. In addition, the distributed training method is combined with the blockchain technology to further benefit the privacy protection, and the proposed blockchain empowered distributed adaptive learning algorithm is applied in vehicular network, which ensures communication security and is immune to attack from a malicious participant. Furthermore, the distributed training framework is extended to reinforcement learning, where deep Q-network is taken as an example. The learning process of deep Q-network is changed into a two-phase update process, where the Q-network of each agent is locally updated based on its own experience first, and the Q-networks of all agents are then globally updated using the consensus algorithm. This allows all agents to learn from other agents' experiences without the sharing of experience samples. Lastly, the distributed deep reinforcement learning framework is applied in the intelligent traffic light control problem, where a group of traffic light agents are connected in a decentralized communication topology. The superiority of the proposed distributed deep Q-networks method for traffic light control is verified by the simulation in SUMO with homogeneous and heterogeneous traffic flow patterns on different intersections.
Date of Award | 1 Aug 2021 |
---|
Original language | English |
---|
Awarding Institution | - The University of Manchester
|
---|
Supervisor | Zhirun Hu (Supervisor) & Zhengtao Ding (Supervisor) |
---|
- Reinforcement Learning
- Neural Network
- Distributed Training
- Consensus
- Machine Learning
Consensus-based Distributed Machine Learning: Theory, Algorithms, and Applications
Liu, B. (Author). 1 Aug 2021
Student thesis: Phd