TY - JOUR
T1 - Safe chance constrained reinforcement learning for batch process control
T2 - A data-driven framework for learning control of uncertain batch process systems
AU - Mowbray, Max
AU - Zhang, Dongda
AU - Petsagkourakis, Panagiotis
AU - del Rio-Chanona, Ehecatl Antonio
PY - 2022/1/1
Y1 - 2022/1/1
N2 - Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independently of explicit assumption of process uncertainty. Recent focus on engineering applications has been directed towards the development of safe RL controllers. Previous works have proposed approaches to account for constraint satisfaction through constraint tightening from the domain of stochastic model predictive control. Here, we extend these approaches to account for plant-model mismatch. Specifically, we propose a data-driven approach that utilizes Gaussian processes for the offline simulation model and use the associated posterior uncertainty prediction to account for joint chance constraints and plant-model mismatch. The method is benchmarked against nonlinear model predictive control via case studies. The results demonstrate the ability of the methodology to account for process uncertainty, enabling satisfaction of joint chance constraints even in the presence of plant-model mismatch.
AB - Reinforcement Learning (RL) controllers have generated excitement within the control community. The primary advantage of RL controllers relative to existing methods is their ability to optimize uncertain systems independently of explicit assumption of process uncertainty. Recent focus on engineering applications has been directed towards the development of safe RL controllers. Previous works have proposed approaches to account for constraint satisfaction through constraint tightening from the domain of stochastic model predictive control. Here, we extend these approaches to account for plant-model mismatch. Specifically, we propose a data-driven approach that utilizes Gaussian processes for the offline simulation model and use the associated posterior uncertainty prediction to account for joint chance constraints and plant-model mismatch. The method is benchmarked against nonlinear model predictive control via case studies. The results demonstrate the ability of the methodology to account for process uncertainty, enabling satisfaction of joint chance constraints even in the presence of plant-model mismatch.
KW - Bioprocess operation
KW - Dynamic optimization
KW - Machine learning
KW - Optimal control
KW - Safe reinforcement learning
U2 - 10.1016/j.compchemeng.2021.107630
DO - 10.1016/j.compchemeng.2021.107630
M3 - Article
SN - 0098-1354
VL - 157
JO - Computers & Chemical Engineering
JF - Computers & Chemical Engineering
M1 - 107630
ER -