eXtended Reality applications for human spaceflight: the ESA-EAC XR Lab

Abstract

The European Space Agency (ESA), and its European Astronaut Centre (EAC), located in Cologne, are currently preparing for human exploration of the Moon and, ultimately, Mars. EAC is the centre of excellence for astronaut training, operations and space medicine with more than 30 years of experience. The centre currently supports International Space Station (ISS) missions, and also prepares for humans to fly again beyond Low Earth Orbit (LEO). To advance human spaceflight, ESA investigates the applications of eXtended Reality (XR) technologies such as Augmented (AR), Virtual (VR) and Mixed Reality (MR). In order to use XR to benefit astronauts, instructors, developers, researchers and operators, the EAC created the XR Lab in 2015. Today, the portfolio of EAC’s XR activities covers missions to the ISS in multiple ways. The XR Lab developed a VR application to train astronauts in manipulating the ISS’s 17 meters-long robotic arm, and familiarize them with the station’s layout. Further XR Lab projects allow to train astronauts for spacewalks in orbit or on a planetary surface. Recently, the XR Lab also started the development of a medical training program to familiarize astronauts with human anatomy. In parallel, the XR Lab covers upcoming human space exploration missions. The next step is the new Lunar Gateway Program, where EAC enables collaborative design reviews in VR. The design of the Lunar Gateway benefits from early reviews of human factors involving astronauts and industries, thanks to high quality simulation of weightlessness in VR. Regarding surface operations, the XR Lab replicated landscapes of the Moon for upcoming robotic missions and outreach activities. Training astronauts to explore the Moon will require new methods, justifying a collaboration between space agencies. The ESA XR Lab, the German Aerospace Center, and the Netherlands Aerospace Centre are cooperating on project focused on advancing training technologies. This cooperation aims to develop and evaluate a new interactive and collaborative astronaut training framework using MR and VR technologies. Future activities will study applications of haptics technologies such as vibrotactiles or exoskeletal devices. The roadmap will also cover virtual rover operations and MR telemedicine support for astronaut flight surgeons. The present paper provides an overview of the XR Lab and how it is exploiting AR, VR and MR technologies to address new exploration challenges related to LEO and beyond.