Optimal Periodic Orbit Stabilization Applied To Elastic Bipedal Locomotion

Abstract

Efficient and robust bipedal locomotion remains a fundamentally challenging control problem due to underactuation, compliance, and contact dynamics. This thesis investigates two different approaches to generating and stabilizing periodic gaits for elastic bipeds. The first approach is purely reinforcement learning based, where both trajectory generation and stabilization are handled jointly. A policy is trained to imitate human motion data and learns robust, energy-efficient gaits through motion imitation and domain randomization. The second approach separates the problem into two stages: trajectory generation, followed by stabilization of the resulting periodic orbit using optimal control, the latter being the focus of this thesis. The stabilization is formulated in transverse coordinates, and an optimal feedback law is computed for the linearized system along the orbit by solving the periodic Riccati equation. To improve the online applicability of the controller, two methods are presented to eliminate the need for coordinate transformations during execution: a Kalman filter-based estimator and a neural network approximation of the value function. A reinforcement learning policy with a structure similar to the optimal controller is also trained for comparison. Both approaches are implemented on the C-Runner platform and compared in terms of robustness and applicability to real-world systems.