DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Smooth Exploration for Robotic Reinforcement Learning

Raffin, Antonin and Kober, Jens and Stulp, Freek (2021) Smooth Exploration for Robotic Reinforcement Learning. In: Conference on Robot Learning (CoRL) 2021. Proceedings of Machine Learning Research. Conference on Robot Learning (CoRL) 2021, 2021, London, UK.

[img] PDF


Reinforcement learning (RL) enables robots to learn skills from interactions with the real world. In practice, the unstructured step-based exploration used in Deep RL -- often very successful in simulation -- leads to jerky motion patterns on real robots. Consequences of the resulting shaky behavior are poor exploration, or even damage to the robot. We address these issues by adapting state-dependent exploration (SDE) to current Deep RL algorithms. To enable this adaptation, we propose two extensions to the original SDE, using more general features and re-sampling the noise periodically, which leads to a new exploration method generalized state-dependent exploration (gSDE). We evaluate gSDE both in simulation, on PyBullet continuous control tasks, and directly on three different real robots: a tendon-driven elastic robot, a quadruped and an RC car. The noise sampling interval of gSDE enables a compromise between performance and smoothness, which allows training directly on the real robots without loss of performance.

Item URL in elib:https://elib.dlr.de/144423/
Document Type:Conference or Workshop Item (Poster)
Title:Smooth Exploration for Robotic Reinforcement Learning
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Raffin, AntoninUNSPECIFIEDhttps://orcid.org/0000-0001-6036-6950UNSPECIFIED
Kober, JensUNSPECIFIEDhttps://orcid.org/0000-0001-7257-5434UNSPECIFIED
Stulp, FreekUNSPECIFIEDhttps://orcid.org/0000-0001-9555-9517UNSPECIFIED
Journal or Publication Title:Conference on Robot Learning (CoRL) 2021
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:No
Publisher:Proceedings of Machine Learning Research
Keywords:Robotics, Reinforcement Learning, Exploration, Real World
Event Title:Conference on Robot Learning (CoRL) 2021
Event Location:London, UK
Event Type:international Conference
Event Dates:2021
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Robotics
DLR - Research area:Raumfahrt
DLR - Program:R RO - Robotics
DLR - Research theme (Project):R - Autonomous learning robots [RO]
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Cognitive Robotics
Deposited By: Raffin, Antonin
Deposited On:29 Nov 2021 11:49
Last Modified:29 Nov 2021 11:49

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.