Exploring Requirements for Neurosurgical Augmented Reality: Design and Evaluation of an Infrared-Based Inside-Out Tracking Approach on HoloLens 2

Abstract

Medical professionals require in-situ visualization of X-ray imaging in 3D to assist with surgical navigation and planning. Research on surgical Augmented Reality (AR) focuses on improving the tracking quality of devices and tools but disregards human requirements. Our user study evaluates application areas and potentials for AR in neurosurgery, as well as feedback for HoloLens 2 based inside-out tracking systems. The main findings include a reported simplification in perception and lowered mental workload. Additionally, we propose a simplified implementation for an infrared-based inside-out rigid body pose estimation system on HoloLens 2 to redirect feedback away from the tracking problem and gain access to human requirements for such systems. Decent patient and exemplary instrument tracking are reached.