A reinforcement learning-based hybrid modeling framework for bioprocess kinetics identification

Constructing predictive models to simulate complex bioprocess dynamics, particularly time-varying (i.e., parameters varying over time) and history-dependent (i.e., current kinetics dependent on historical culture conditions) behavior, has been a longstanding research challenge. Current advances in hybrid modeling offer a solution to this by integrating kinetic models with data-driven techniques. This article proposes a novel two-step framework: first (i) speculate and combine several possible kinetic model structures sourced from process and phenomenological knowledge, then (ii) identify the most likely kinetic model structure and its parameter values using model-free Reinforcement Learning (RL). Specifically, Step 1 collates feasible history-dependent model structures, then Step 2 uses RL to simultaneously identify the correct model structure and the time-varying parameter trajectories. To demonstrate the performance of this framework, a range of in-silico case studies were carried out. The results show that the proposed framework can efficiently construct high-fidelity models to quantify both time-varying and history-dependent kinetic behaviors while minimizing the risks of over-parametrization and over-fitting. Finally, the primary advantages of the proposed framework and its limitation were thoroughly discussed in comparison to other existing hybrid modeling and model structure identification techniques, highlighting the potential of this framework for general bioprocess modeling.