Development of controller using Reinforcement learning with Short term PV forecast for grid voltage stability

Abstract

Ensuring voltage stability in power grids with high photovoltaic (PV) penetration is a critical challenge due to the intermittent nature of solar energy. This thesis presents the development of a reinforcement learning (RL)-based voltage control framework that leverages short-term PV forecasts to enhance grid stability. The research focuses on integrating machine learning-driven voltage forecasting with RL-based decision-making, executed within a co-simulation framework that connects MATLAB-based grid simulations with Python-based control logic using Mosaik.

A machine learning model was developed to predict grid voltage fluctuations based on forecasted PV power generation and load demand. The goal was to provide an accurate forecast of future voltage states to inform the RL-based controller. The co-simulation framework was established to enable real-time data exchange between MATLAB and Python, allowing the RL algorithm to make informed voltage control decisions. However, despite the structured approach, several technical challenges hindered the successful execution of the RL-based control system. Synchronization mismatches between MATLAB and Python, inconsistencies in data exchange, and computational constraints due to multi-threading and queue-based communication prevented the RL controller from receiving complete state observations, thereby rendering training infeasible. These challenges highlight the complexity of implementing real-time reinforcement learning for voltage stability control.

The findings from this research underscore the necessity of improved synchronization mechanisms, buffering techniques, and refined RL training methodologies to better integrate and train RL-based voltage control within the co-simulation platform. Future research directions include the implementation of enhanced synchronization strategies, offline pre-training of RL models using historical data, and the incorporation of additional predictive features into the voltage forecasting model. Furthermore, alternative control strategies, such as hybrid AI-based controllers that integrate rule-based decision-making with reinforcement learning, could improve reliability and performance.

This research contributes to the growing body of work on AI-driven grid stability solutions by demonstrating the challenges and potential improvements in reinforcement learning-based voltage control within the co-simulation framework. Addressing these challenges through enhanced co-simulation frameworks and refined control methodologies will be crucial for improving our co-simulation approach, enabling seamless transitions between software-based and hardware-based setups for voltage control algorithm development.