Assisting engineering, training and operations for human spaceflight applications using eXtended Reality (XR) technologies

Abstract

The European Astronaut Centre (EAC) located in Cologne, Germany, is the centre of excellence for human spaceflight. It is responsible for astronaut and ground controllers training, operations and space medicine in service of human exploration of low Earth orbit (LEO), the Moon and eventually Mars. In 2015, the centre established the XR Lab, a facility that uses eXtended Reality (XR) technologies such as Augmented (AR), Virtual (VR) and Mixed Reality (MR) to benefit astronauts, instructors, researchers and operators. Under the European Space Agency’s (ESA) lead, the XR Lab is developing projects at EAC in collaboration with the German Aerospace Center (DLR), other space agencies as well as partners from the industry and academia. Various XR projects developed at EAC support current missions to the International Space Station (ISS). VR and MR are used to help astronauts to familiarize with the ISS architecture and layout, learn how to fly the ISS’s robotic arm and practice Intra (IVAs) and Extravehicular Activities (EVAs) supported by a microgravity physics engine. Technology demonstration experiments have also been conducted directly onboard ISS to test the use of VR devices in microgravity for on board training tasks. Moreover, a custom in-house VR headset is being developed to address the challenges of motion tracking in microgravity. VR is also used for training at analogue sites, such as outdoor natural landscapes of the ESA CAVES and PANGAEA programmes or indoor facilities, such as the upcoming ESA-DLR LUNA analogue facility. Additionally, the EAC XR Lab is involved in the development of virtual engineering models and digital twins for concept-of-operations and hardware design. Using VR and MR collaborative tools, design reviews, habitability tests and user studies have been conducted for the Lunar Gateway program and for the future ESA Argonaut lander. Immersing users in high-fidelity simulations that replicate relevant environmental conditions, such as hypogravity and lunar lighting, allows for the collection of valuable insights and actionable feedback. This marks an important improvement over the capabilities of traditional non-immersive methods commonly employed in the space sector. Lastly, future activities are looking into the integration of haptics technologies, such as vibrotactiles or exoskeletal devices, as well as virtual rover operations, MR-based astronaut flight surgeons’ support tools and the use of VR for underwater applications in neutral buoyancy facilities. The presentation will provide an overview of the XR Lab portfolio that uses AR, VR and MR technologies to support the human exploration of the solar system.