DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Closing the Loop: 3D Object Tracking for Advanced Robotic Manipulation

Stoiber, Manuel (2023) Closing the Loop: 3D Object Tracking for Advanced Robotic Manipulation. Dissertation, Technische Universität München.

[img] PDF

Official URL: https://mediatum.ub.tum.de/1696259


In robotics, it is often assumed that the world is static, forward kinematics are perfectly known, and interactions are fully deterministic. However, while those assumptions are true for some applications, in general, they significantly limit the complexity of manipulation tasks and the design of robotic hardware. To overcome such limitations, this thesis focuses on 3D object tracking techniques for advanced robotic manipulation. The goal is to develop a flexible and efficient algorithm that uses camera data to provide continuous pose estimates for the robot's end effector and relevant objects. Given such feedback, it is possible to adopt more human-like visual servoing approaches, react to changes in the environment, and facilitate safe and efficient robot designs. The first main contribution of this work is a highly-efficient approach to region-based object tracking. It features a well-founded probabilistic model that considers image information sparsely along so-called correspondence lines. Experiments demonstrate that the algorithm performs significantly better than existing region-based methods while being approximately one order of magnitude faster. Subsequently, based on this approach, an extension to multi-modality tracking is discussed. The method allows to fuse depth, texture, and multi-region information from various cameras in a highly-modular probabilistic formulation. Again, experiments on different datasets show that the resulting algorithm is highly efficient and outperforms both conventional and deep learning-based methods by a considerable margin. Finally, the third part of this thesis considers the tracking of multi-body objects, which are composed of rigid bodies that are connected by joints. For this, a flexible framework is proposed that allows the extension of existing rigid object tracking methods to multi-body tracking. It considers both tree-like and closed kinematic structures and facilitates the flexible configuration of joints and constraints. For a detailed evaluation, a highly-realistic synthetic dataset is introduced that features a large number of sequences and various robots. Experiments demonstrate the excellent performance of the developed framework and tracker. All theoretical concepts are implemented into the multi-body, multi-modality, and multi-camera tracking library M3T, which we released as open-source software. It provides a highly-modular architecture that supports a wide range of kinematic structures, object characteristics, and camera setups. In addition, the algorithm allows the incorporation of robot joint measurements of varying accuracy and reliability. In real-world experiments on the humanoid robot David and the MiroSurge system, it is used to continuously provide the pose and configuration of manipulated objects and robot end effectors. The developed approach thereby allows to close the perception-action loop and facilitates new capabilities for advanced robotic manipulation.

Item URL in elib:https://elib.dlr.de/203345/
Document Type:Thesis (Dissertation)
Title:Closing the Loop: 3D Object Tracking for Advanced Robotic Manipulation
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Stoiber, ManuelUNSPECIFIEDhttps://orcid.org/0000-0002-0762-9288UNSPECIFIED
Date:20 December 2023
Open Access:Yes
Number of Pages:172
Keywords:object tracking
Institution:Technische Universität München
Department:Department of Computer Engineering
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Multisensory World Modelling (RM) [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Institute of Robotics and Mechatronics (since 2013)
Deposited By: Strobl, Dr. Klaus H.
Deposited On:20 Mar 2024 08:27
Last Modified:20 Mar 2024 08:27

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.