Control and Optimization of Chemical Reactors with Model-free Deep Reinforcement Learning

Abstract: Model-based control and optimization is the predominant paradigm in process systems engineering. The performance of model-based methods, however, rely heavily on the accuracy of the process model, which declines over the operation cycle due to various causes, such as catalyst deactivation, equipment aging, feedstock variability, and others. This work aims to tackle this challenge by considering two alternative approaches. The first approach replaces existing control and optimization methods with model-free reinforcement learning (RL). We apply a state-of-the-art reinforcement learning algorithm to a network of reactions, evaluate the performance of the RL controller in terms of setpoint tracking, disturbance rejection, and robustness to parameter uncertainties, and optimize the reward function to achieve the desired control and optimization performance. The second approach presents a novel framework for integrating Economic Model Predictive Control (EMPC) and RL for online model parameters estimation. In this framework, EMPC optimally operates the closed-loop system while maintaining closed-loop stability and recursive feasibility. At the same time, the RL agent continuously compares the measured state of the process with the model’s predictions, and modifies the model parameters accordingly to optimize the process. The performance of the proposed framework is illustrated on a network of reactions with challenging dynamics and practical significance.