Enhancing Scalable Autonomy Space Teleoperation with User Intervention During Task Execution

Kurzfassung

With increasing task complexity in space robotic mission designs, advancement in the command technology and remote operation of the robotic team becomes a key challenge for providing an effective human-robotic team interface to help facilitate successful task execution. In the International Space Station (ISS)-to-ground telerobotic experiments, Surface Avatar led by DLR with partner ESA, a team of robots on the (Earth) surface is commanded by an astronaut in orbit to perform various tasks in a simulated space habitat. The ISS crew can select different input modalities to command the robots with direct teleoperation with the aid of force reflection in some instances, which provides great user immersion and interaction with the environment. At the other end of the Scalable Autonomy spectrum, the robot team can also be commanded at the task level, utilizing the robot's local intelligence to plan and execute at the crew's high-level direction. By scaling and mixing these telerobotic command capabilities, we can bring human and robot intelligence together to solve and execute more complex and unknown tasks. The teleoperator, whether an astronaut or ground-based expert, would be able to effectively command a surface robotic team in a wide variety of conditions. A concern raised for autonomously executed robotic task is the ability to cope with unexpected situations in a timely manner. To help enhance this command scalability, particularly for a more seamless transition between the command modes, the current research presented in this paper introduces a system that allows the crew to intervene with direct user input to provide course correction during autonomous task execution mid-task execution. This work explores different solutions to provide the human teleoperator with situational awareness of the robot's action, and approaches for user intervention. One approach we developed provides guidance forces based on the autonomous task and uses the movement of the haptic device as a comparison metric for interpreting the user intention. The user can observe the robot's action via the video stream, and let the robot complete the task autonomously. The robot executes the tasks autonomously as long as the robot's planned action (e.g. motion) concurs with user's intention. Should the need or desire arise, the user can take over with direct teleoperation at any moment using the haptic interface. The haptic cues provided by the haptic input device can take different forms such as pose, position, or velocity. In order to understand the effectiveness of these different approaches for allowing the teleoperator to take over the robot's autonomous task, particularly for space deployment, an implementation for user intervention is tested during the Surface Avatar ISS-Earth telerobotic experiment in 2025 where the ISS crew commanded a team of robots to perform various sample handling tasks with Scalable Autonomy teleoperation. Its outcome and astronaut feedback are discussed. Furthermore, a user study is carried out on-ground in the same simulated space habitat with the time-delay conditions experienced by the ISS crew to study the performance and usability of seamless switching of telecommand modalities in Scalable Autonomy teleoperation.