Learning-Based Class-Agnostic Segmentation of Grasped Objects

Abstract

Autonomous robotic applications require the ability to acquire information about newly encountered objects. However, currently most methods are trained in an offline manner, and require a large amount of annotated data. To meet the respective demand, this thesis proposes two methods to automatically obtain binary labels of grasped objects. Inspired by human learning through interaction, any object instance can be labeled just by concurrent manual manipulation and observation. Specifically, two approaches are explored: First, the presence of a hand in an image is leveraged as a cue to guide a network for the task of object-in-hand segmentation. A diverse set of labeled object instances enables the model to learn an object-agnostic item-in-hand representation. Second, a hand or robot manipulator representation is learned by exploiting motion cues as optical flow. Given a static scene, a moving grasped object can then be differentiated from the robot or hand for the segmentation of the grasped object. Through extensive experimental evaluation, the effectiveness of both methods under realistic conditions is verified.