Vision-guided Grasping of Arbitrary Objects through Experience-based Search Optimization

Abstract

A desired capability for human-robot collaboration is the hand over of tools and object parts in a functional, effective way. Therefore, a robot has to grasp objects at specific spots, also called functional grasps. In this thesis an approach for such functional grasping of nearly unknown, arbitrary objects is presented. A differentiating factor to other approaches is the lack of a requirement for 3D models of the objects. Given a camera mounted on the robot's end-effector, the grasping position is defined by a single target image with human guidance. During execution, an iterative search for the target viewing angle is performed, based on an appearance similarity measure generated by an Auto-Encoder (AE) specifically trained to encode general object rotation. Further, a process for the fusion of data from previous grasping attempts is presented. This increases robustness and search efficiency by utilizing experience from previous executions. Additionally, a detection module is integrated, which enables the grasping of the target object in cluttered scenes. The developed method is evaluated in a simulation and on a real robotic platform. It can be shown, that the presented method is able to robustly find the pre-defined target orientation to grasp the objects.