Closing the Gap: Combining Task Specification and Reinforcement Learning for Compliant Vegetable Cutting

Abstract

Easy-to-define but flexible behaviors are key to the successful deployment of service robots in more everyday situations. Complex modeling of the world dynamics is infeasible outside of very controlled environments like manufacturing cells. Learning new behaviors has the advantage of a high flexibility, but comes at the price of prohibitively many learning iterations when applied on a real robot system. We combine learning with a simple task specification to guide the learning process with human knowledge about the problem. As application, we demonstrate a compliant manipulation task using reinforcement learning guided by the Task Frame Formalism. This allows us to specify the easy to model knowledge about a task while the robot learns the unmodeled components by reinforcement learning. We evaluate the approach by cutting vegetables with a KUKA LWR 4+ manipulator. The robot was able to learn force control policies directly on the robot without using any simulation.