Safe and Efficient Control of a Brayton Cycle Heat Pump Using Reinforcement Learning

Abstract

Decarbonizing industrial process heating will increasingly depend on high-temperature heat pumps. In particular, Brayton cycle heat pumps, which can reach temperatures above 250 °C, are viewed as a promising technology. However, ensuring safe operation and optimal control remains challenging. This study presents an experimentally validated dynamic model of a Brayton cycle heat pump, a system with multiple control inputs for regulating its thermal output. Using this model as a training environment, several control concepts integrating Reinforcement Learning (RL) and traditional PI controllers were implemented to achieve desired heat supply at target temperatures. Domain randomization was employed to improve the controller robustness against model uncertainties in preparation for deployment on the physical system. The results demonstrate that RL controllers can not only achieve the desired set-point temperature under varying loads while maintaining required safety margins, but also discovered a novel, more energy-efficient operational strategy.