Reinforcement Learning for Heliostat Control in Solar Tower Power Plants

Abstract

Solar thermal tower power plants present a promising solution for scalable renewable energy. These power plants concentrate sunlight onto a central receiver using a field of controllable mirrors called heliostats. However, due to the nature and scale of these fields, even slight misalignments or mechanical imperfections result in significant losses in absorbed power. Traditional open-loop control strategies require extensive calibration of each individual heliostat, which is time inefficient and can take anywhere between a few weeks and a few months, and drives up operational cost. Recent work demonstrates the ability of model-free RL methods to dynamically distribute heliostat aim-points on the receiver’s surface and achieve substantial improvements in terms of annual absorbed power. Despite the promising results, model-free RL methods suffer from high sample inefficiency and do not converge reliably. In this thesis, we address the more difficult task of directly controlling heliostat orientations. By leveraging analytical gradients from a differentiable simulator, our agents not only exhibit sample-efficient and reliable convergence but also outperform model-free RL methods and model predictive control.