Boerdijk, Wout and Sundermeyer, Martin and Durner, Maximilian and Triebel, Rudolph (2021) "What's This?" - Learning to Segment Unknown Objects from Manipulation Sequences. In: 2021 IEEE International Conference on Robotics and Automation, ICRA 2021. IEEE Robotics and Automation Society. 2021 IEEE International Conference on Robotics and Automation, ICRA 2021, 2021-05-31 - 2021-06-05, Xi'an, China / online (hybrid). doi: 10.1109/ICRA48506.2021.9560806. ISBN 978-172819077-8. ISSN 1050-4729.
PDF
4MB |
Abstract
We present a novel framework for self-supervised grasped object segmentation with a robotic manipulator. Our method successively learns an agnostic foreground segmentation followed by a distinction between manipulator and object solely by observing the motion between consecutive RGB frames. In contrast to previous approaches, we propose a single, end-toend trainable architecture which jointly incorporates motion cues and semantic knowledge. Furthermore, while the motion of the manipulator and the object are substantial cues for our algorithm, we present means to robustly deal with distraction objects moving in the background, as well as with completely static scenes. Our method neither depends on any visual registration of a kinematic robot or 3D object models, nor on precise hand eye calibration or any additional sensor data. By extensive experimental evaluation we demonstrate the superiority of our framework and provide detailed insights on its capability of dealing with the aforementioned extreme cases of motion. We also show that training a semantic segmentation network with the automatically labeled data achieves results on par with manually annotated training data. Code and pretrained model are available at https://github.com/DLR-RM/DistinctNet.
Item URL in elib: | https://elib.dlr.de/146603/ | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Document Type: | Conference or Workshop Item (Speech) | ||||||||||||||||||||
Title: | "What's This?" - Learning to Segment Unknown Objects from Manipulation Sequences | ||||||||||||||||||||
Authors: |
| ||||||||||||||||||||
Date: | 2021 | ||||||||||||||||||||
Journal or Publication Title: | 2021 IEEE International Conference on Robotics and Automation, ICRA 2021 | ||||||||||||||||||||
Refereed publication: | Yes | ||||||||||||||||||||
Open Access: | Yes | ||||||||||||||||||||
Gold Open Access: | No | ||||||||||||||||||||
In SCOPUS: | Yes | ||||||||||||||||||||
In ISI Web of Science: | Yes | ||||||||||||||||||||
DOI: | 10.1109/ICRA48506.2021.9560806 | ||||||||||||||||||||
Publisher: | IEEE Robotics and Automation Society | ||||||||||||||||||||
ISSN: | 1050-4729 | ||||||||||||||||||||
ISBN: | 978-172819077-8 | ||||||||||||||||||||
Status: | Published | ||||||||||||||||||||
Keywords: | Deep Learning for Visual Perception, Semantic Segmentation, Computer Vision for Automation | ||||||||||||||||||||
Event Title: | 2021 IEEE International Conference on Robotics and Automation, ICRA 2021 | ||||||||||||||||||||
Event Location: | Xi'an, China / online (hybrid) | ||||||||||||||||||||
Event Type: | international Conference | ||||||||||||||||||||
Event Start Date: | 31 May 2021 | ||||||||||||||||||||
Event End Date: | 5 June 2021 | ||||||||||||||||||||
HGF - Research field: | Aeronautics, Space and Transport | ||||||||||||||||||||
HGF - Program: | Space | ||||||||||||||||||||
HGF - Program Themes: | Robotics | ||||||||||||||||||||
DLR - Research area: | Raumfahrt | ||||||||||||||||||||
DLR - Program: | R RO - Robotics | ||||||||||||||||||||
DLR - Research theme (Project): | R - Multisensory World Modelling (RM) [RO] | ||||||||||||||||||||
Location: | Oberpfaffenhofen | ||||||||||||||||||||
Institutes and Institutions: | Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition | ||||||||||||||||||||
Deposited By: | Boerdijk, Wout | ||||||||||||||||||||
Deposited On: | 08 Dec 2021 14:22 | ||||||||||||||||||||
Last Modified: | 24 Apr 2024 20:45 |
Repository Staff Only: item control page