Enhancing Manipulation Performance with Grounded Haptic UI Teleoperation

Abstract

Teleoperation refers to the process of remotely controlling a robot by a human operator. The objective of teleoperation is to complete tasks in inaccessible or hazardous environments while under human supervision. Within teleoperation, manipulating objects in the environment can be particularly challenging because the human operator has limited feedback compared to the robot. Despite the robot having access to depth information, the human operator may only have access to 2D visuals or a limited representation of 3D data, depending on network bandwidth and time lag. One potential solution is to enable the robot to guide the human operator as the robot has access to a higher level of feedback. With assistance from the robot, teleoperation can potentially improve safety, efficiency, and performance in complex environments. This thesis investigates a framework for teleoperation that combines depth information and force feedback to enable intuitive and natural manipulation of objects by a human operator. The framework will implement the Elastic Strip framework by Oliver Brock to provide guidance to the operator and convey the desired trajectory for object grasping and manipulation through force feedback. Additionally, uncertainty in obstacle positions will be considered, and probabilistic algorithms will be used to plan a dynamic, collision-free trajectory for grasping. A grounded exoskeleton is integrated as a user interface to provide force feedback to the operator, enhancing the user situational awareness and helping to improve the performance of teleoperated manipulation tasks. The performance of the proposed framework will be evaluated through a series of experiments to analyze the effectiveness of the shared control approach in enabling accurate and efficient manipulation of objects in a variety of scenarios. These scenarios range from simple pick and place tasks to more complex assembly tasks, such as peg-in-hole. The context of application in this project is space exploration, with the eventual goal of assisting astronauts in teleoperation tasks. Additional methodologies, including reinforcement learning, will also be explored to further enhance the efficiency of the system. This will involve identifying scenarios where the assistive algorithm fails, and the expert, in this case, the operator, takes over the task. This will eventually lead to a smarter system over time. Overall, this thesis presents a novel framework for teleoperated manipulation that incorporates depth information, force feedback, and a grounded exoskeleton to enable intuitive and effective object manipulation by a human operator.