DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation

Röstel, Lennart and Pitz, Johannes and Sievers, Leon and Bäuml, Berthold (2024) Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation. In: 22nd IEEE-RAS International Conference on Humanoid Robots, Humanoids 2023, pp. 1-8. IEEE. 2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids), 2023-12-12 - 2023-12-14, Austin, TX, USA. doi: 10.1109/Humanoids57100.2023.10375194. ISBN 979-835030327-8. ISSN 2164-0572.

Full text not available from this repository.

Official URL: https://ieeexplore.ieee.org/document/10375194


This paper identifies and addresses the problems with naively combining (reinforcement) learning-based controllers and state estimators for robotic in-hand manipulation. Specifically, we tackle the challenging task of purely tactile, goal-conditioned, dextrous in-hand reorientation with the hand pointing downwards. Due to the limited sensing available, many control strategies that are feasible in simulation when having full knowledge of the object's state do not allow for accurate state estimation. Hence, separately training the controller and the estimator and combining the two at test time leads to poor performance. We solve this problem by coupling the control policy to the state estimator already during training in simulation. This approach leads to more robust state estimation and overall higher performance on the task while maintaining an interpretability advantage over end-to-end policy learning. With our GPU-accelerated implementation, learning from scratch takes a median training time of only 6.5 hours on a single, low-cost GPU. In simulation experiments with the DLR-Hand II and for four significantly different object shapes, we provide an in-depth analysis of the performance of our approach. We demonstrate the successful sim2real transfer by rotating the four objects to all 24 orientations in the pi/2 discretization of SO(3), which has never been achieved for such a diverse set of shapes. Finally, our method is able to reorient a cube consecutively to in median nine goals, which was beyond the reach of previous methods in this challenging setting. (Web: https://dlr-alr.github.io/dlr-tactile-manipulation).

Item URL in elib:https://elib.dlr.de/202624/
Document Type:Conference or Workshop Item (Speech)
Title:Estimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Röstel, LennartUNSPECIFIEDhttps://orcid.org/0000-0002-3397-0658UNSPECIFIED
Pitz, JohannesUNSPECIFIEDhttps://orcid.org/0000-0002-2629-1892UNSPECIFIED
Sievers, LeonUNSPECIFIEDhttps://orcid.org/0000-0001-6430-4618UNSPECIFIED
Bäuml, BertholdUNSPECIFIEDhttps://orcid.org/0000-0002-4545-4765UNSPECIFIED
Date:1 January 2024
Journal or Publication Title:22nd IEEE-RAS International Conference on Humanoid Robots, Humanoids 2023
Refereed publication:Yes
Open Access:No
Gold Open Access:No
In ISI Web of Science:No
Page Range:pp. 1-8
Keywords:reinforcement learning
Event Title:2023 IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)
Event Location:Austin, TX, USA
Event Type:international Conference
Event Start Date:12 December 2023
Event End Date:14 December 2023
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Autonomy & Dexterity [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013)
Deposited By: Strobl, Dr. Klaus H.
Deposited On:05 Feb 2024 08:48
Last Modified:24 Apr 2024 21:02

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.