DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Adaptive Model Mediated Control Using Reinforcement Learning

Beik-Mohammadi, Hadi (2020) Adaptive Model Mediated Control Using Reinforcement Learning. Master's, Universität Hamburg.

[img] PDF


Due to similarities in learning techniques, Reinforcement Learning (RL) is the closest alternative to human-level intelligence. Teleoperation systems using RL can adapt to new environmental conditions and deal with high uncertainty due to long-time delays. In this thesis, we propose a method that takes advantage of RL capabilities to extend the human reach in dangerous remote environments. The proposed method utilizes the Model Mediated Teleoperation (MMT) concept in which the teleoperator interacts with a simulated setup that resembles the real environment. The simulation can provide instant haptic feedback where the data from the real environment are delayed. The proposed approach enables haptic feedback teleoperation of high-DOF dexterous robots under long time delays in a time-varying environment with high uncertainty. In existence of time delay, when the data is received by the remote system the environment may change drastically, therefore, the attempt for task execution will fail. To prevent failure, an intelligence system is realized in two layers, the first layer utilizes the Dynamic Movement Primitives (DMP) which accounts for certain changes in the environment. DMPs can adjust the shape of a trajectory based on given criteria, for example, a new target position or avoiding a new obstacle. But in an uncertain environment, DMPs fail, therefore, the second layer of intelligence makes use of different reinforcement learning methods based on expectation-maximization, stochastic optimal control and policy gradient to guarantee the successful completion of the task. Furthermore, To ensure the safety of the system, and speed up the learning process, each learning session for RL happens in multiple simulations of the remote system and environment, simultaneously. The proposed approach was realized on DLR's haptic hand-arm user interface/exoskeleton, Exodex Adam. It has been used for the first time in this work as the master device to teleoperate a high-DOF dexterous robot. This slave device is an anthropomorphic hand-arm system combining a five-finger hand (FFH) attached to a custom configured DLR lightweight robot (LWR 4+) more closely fitting to the kinematics of the human arm. An augmented reality visualization implemented on the Microsoft Hololens fuses the slave device and virtual environment models to provide environment immersion for the teleoperator. A preliminary user-study was carried out to help evaluate the human-robot interaction capabilities and performance of the system. Meanwhile, the RL approaches are evaluated separately in two different levels of difficulty; with and without uncertainty in perceived object position. The results from the unweighted NASA Task load Index (NASA TLX) and System Usability Score (SUS) questionnaires show a low workload (27) and above-average perceived usability (71). The learning results show all RL methods can find a solution for all challenges in a limited time. Meanwhile, the method based on stochastic optimal control has a better performance. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved.

Item URL in elib:https://elib.dlr.de/139999/
Document Type:Thesis (Master's)
Additional Information:Supervisors: Prof. Dr. S. Wermter, Universität Hamburg Dr. Matthias Kerzel, Universität Hamburg Dr. Neal Y. Lii, DLR Award: Universität Best Master's Thesis Award, 2020
Title:Adaptive Model Mediated Control Using Reinforcement Learning
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Journal or Publication Title:Adaptive Model Mediated Control Using Reinforcement Learning
Refereed publication:No
Open Access:Yes
Number of Pages:84
Keywords:Machine Learning and Adaptation; Model Mediated Control; Virtual and Augmented Tele-presence Environments; Novel Interfaces and Interaction Modalities; Telerobotics; Human-Robot Interaction
Institution:Universität Hamburg
Department:Department Informatik
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Space System Technology
DLR - Research area:Raumfahrt
DLR - Program:R SY - Space System Technology
DLR - Research theme (Project):R - Telerobotics (old)
Location: Oberpfaffenhofen
Institutes and Institutions:Institute of Robotics and Mechatronics (since 2013) > Autonomy and Teleoperation
Deposited By: Lii, Neal Yi-Sheng
Deposited On:04 Jan 2021 10:04
Last Modified:04 Jan 2021 10:04

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.