Verification and Validation of Reinforcement Learning based Aeroelastic Control System

Kurzfassung

This paper presents the adaptation of established Verification & Validation (V&V) practices for classical flight control to a data-driven Reinforcement Learning (RL) based Gust Load Alleviation (GLA) control system, with the aim of addressing the simulation to reality gap in RL based aeroelastic control applications. The control law was developed using the Soft ActorCritic (SAC) algorithm as an end-to-end deep neural network that maps sensor measurements to actuator commands, and was trained offline using data from a linear Aeroservoelastic (ASE) model. The resulting fixed-policy controller is then subjected to a V&V workflow comprising closed-loop verification through simulation analyses on a high-fidelity nonlinear ASE model, followed by experimental validation through wind tunnel testing. Validation was conducted on a flexible wing demonstrator under harmonic gust excitation generated by cylindrical gust generators. Performance was evaluated against a set of control design specifications that cover nominal load alleviation, robustness to unstructured and parametric uncertainties, actuator and safety constraints. Wind tunnel results demonstrated that the RL controller attained a maximum Wing Root Bending Moment (WRBM) reduction of 65% and 80% using single and dual-actuator configurations, respectively, at the first wing bending mode frequency.