Extending the Hybrid Agent for Reinforcement Learning Beyond Fixed-Length Scenarios

Abstract

In Quantum Reinforcement Learning, the "hybrid agent for quantum-accessible reinforcement learning" (Hamann and Wölk, 2022) provides a quadratic speed-up in terms of sample complexity over classical algorithms. This hybrid agent may be used in deterministic and strictly episodic environments, for which the maze problem is a standard example. With the current algorithm, however, the episode length (i.e., the number of actions to be played in an episode) is a hyperparameter which needs to be set. For scenarios such as mazes with an unknown distance towards the goal, this poses a problem, since a feasible episode length value is not known initially. In this work, we propose an adaption to the hybrid algorithm that uses a variable episode length selection strategy, allowing its usage in a wider range of maze problem scenarios. We test our novel approach against classical agents in various maze scenarios. Finally, we reason about conditions for which a quantum advantage persists.