Human-supported Robotic Perception for Autonomous Manipulation

Kurzfassung

Perception is a crucial part of a robotic system because understanding the environment is the basis for successful completion of all kinds of tasks. While robots should be able to cope with a wide variety of environmental conditions fully autonomously in the long run, their capabilities still do not reach those of their human counterparts. Especially unstructured and priorly unknown environments e.g. found in space missions as well as household scenarios still pose big challenges for today's perception algorithms. Given that humans naturally deal with these environments on a regular basis, choosing a shared autonomy approach in which a user can supplement the robot's perception, can offer promising improvements. This work investigates the use of a novel Graphical User Interface enabling users to support a robot in the detection of objects and their poses as for instance required for autonomous manipulation tasks. The developed prototype allows users to manipulate virtual object meshes in a 3D environment in order to align them with the equivalent objects in the real world. The sensor data of the robot's surroundings is visualized through a point cloud supplemented by existing autonomous object detections in order to provide orientation to the user during the task. An initial user study, considering both the User Experience as well as the general performance and accuracy across scenarios of varying difficulty, was conducted aiming to examine the GUI's feasibility. Additionally, suitable input modalities were explored to ensure an efficient and pleasant usage. The chosen approach allows for precise alignments while also proving a good usability. In the process, high occlusions and especially the point cloud's quality have been identified as critical factors in regards to pose quality as well as the users' confidence. Furthermore, the advantages of a handle-based concept for DoF separation are highlighted in regards to the manipulation of the virtual objects. In addition, the system was successfully tested on the humanoid robot Rollin' Justin for experimental object manipulation tasks.