DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities

Lee, Jongseok and Radhakrishna Balachandran, Ribin and Kondak, Konstantin and Coelho, Andre and De Stefano, Marco and Humt, Matthias and Feng, Jianxiang and Asfour, Tamim and Triebel, Rudolph (2023) Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities. Field Robotics, 3, pp. 323-367. Field Robotics Publication Society. doi: 10.55417/fr.2023010. ISSN 2771-3989.

[img] PDF - Published version

Official URL: https://fieldrobotics.net/Field_Robotics/Volume_3_files/Vol3_10.pdf


This paper presents a novel telepresence system for advancing aerial manipulation in dynamic and unstructured environments. The proposed system not only features a haptic device, but also a virtual reality (VR) interface that provides real-time 3D displays of the robot's workspace as well as a haptic guidance to its remotely located operator. To realize this, multiple sensors, namely, a LiDAR, cameras, and IMUs are utilized. For processing of the acquired sensory data, pose estimation pipelines are devised for industrial objects of both known and unknown geometries. We further propose an active learning pipeline in order to increase the sample efficiency of a pipeline component that relies on a Deep Neural Network (DNN) based object detector. All these algorithms jointly address various challenges encountered during the execution of perception tasks in industrial scenarios. In the experiments, exhaustive ablation studies are provided to validate the proposed pipelines. Methodologically, these results commonly suggest how an awareness of the algorithms' own failures and uncertainty ("introspection") can be used to tackle the encountered problems. Moreover, outdoor experiments are conducted to evaluate the effectiveness of the overall system in enhancing aerial manipulation capabilities. In particular, with flight campaigns over days and nights, from spring to winter, and with different users and locations, we demonstrate over 70 robust executions of pick-and-place, force application and peg-in-hole tasks with the DLR cable-Suspended Aerial Manipulator (SAM). As a result, we show the viability of the proposed system in future industrial applications.

Item URL in elib:https://elib.dlr.de/194517/
Document Type:Article
Title:Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Lee, JongseokUNSPECIFIEDhttps://orcid.org/0000-0002-0960-0809UNSPECIFIED
Radhakrishna Balachandran, RibinUNSPECIFIEDhttps://orcid.org/0000-0002-7560-471XUNSPECIFIED
Coelho, AndreUNSPECIFIEDhttps://orcid.org/0000-0002-0917-5574UNSPECIFIED
Humt, MatthiasUNSPECIFIEDhttps://orcid.org/0000-0002-1523-9335UNSPECIFIED
Asfour, TamimInstitute for Anthropomatics and RoboticsUNSPECIFIEDUNSPECIFIED
Date:1 March 2023
Journal or Publication Title:Field Robotics
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:No
Page Range:pp. 323-367
Publisher:Field Robotics Publication Society
Keywords:pose estimation, active learning, virtual reality, telepresence, aerial manipulation, uncertainty quantification, introspection
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Explainable Robotic AI
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Institute of Robotics and Mechatronics (since 2013) > Perception and Cognition
Deposited By: Lee, Jongseok
Deposited On:29 Mar 2023 11:15
Last Modified:29 Mar 2023 11:15

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.