Enhancing Probabilistic Imitation Learning with Robotic Perception For Self-Organising Robot Workstation

Abstract

The increasing adoption of robotic automation across various industries is driving extensive research into efficient and intuitive methods for transferring skills from humans to robots. Traditional robot programming techniques, along with the expertise they require, remain significant obstacles to scalability of robotic deployment and integration. Learning from human demonstrations has emerged as a promising approach to teach robots complex behaviors while making robot programming more accessible. For this approach to be effective, the demonstration process must be user-friendly, and the imitation learning algorithms must reliably generate the movements necessary to complete tasks in current and future manufacturing contexts. To address these needs, we develop a demonstration recording framework and combine it with state-of-the-art techniques from imitation learning and computer vision, forming an end-to-end software pipeline for teaching and deploying robotic skills. Our system is implemented on the Safe Autonomous Robotic Assistant (SARA) robot to record kinesthetic demonstrations and teach it to self-organize its workstation by picking up scattered objects. We utilize Kernelized Movement Primitives (KMP) to learn the movements and adapt them to different environment configurations by adding start and end poses as via points. A You Only Look Once (YOLO) vision model is trained to detect objects on the workstation, and the 3D pose is estimated from the 2D bounding box. This vision system serves two primary functions: first, it aids in automatically recording object-centric demonstrations, eliminating the need for manual object pose determination by the programmer; second, it ensures precise interaction with objects regardless of their pose on the workstation. During deployment, it helps generalize the learned task by providing the pose of the objects to be grasped. The developed system achieves reliable detections and pose estimations for a variety of object poses on the workstation, which the robot is able to clean up effectively and autonomously.