Human-AI Teaming - Challenges from a Practitioner's Perspective

Abstract

The integration of Artificial Intelligence (AI) in aviation is gaining momentum, with potential benefits for pilots, air traffic controllers, and airport operations. However, the adoption of AI in safety-critical tasks poses dilemmas for developers and researchers, who must balance the need for proper processes and standards with the complexities of human-AI teaming. Recent research has highlighted the importance of transparency, explainability, and trust in AI systems. When in the future, AI collaborates with the human operator to work towards a shared goal, both the AI and the human need to communicate their motivations and consideration, which poses a different challenge from other applications with higher levels of automation. The EU-funded SESAR project JARVIS has explored these challenges through a workshop with digital assistant designers, identifying key questions such as the need for human-AI teaming versus using AI as a mere tool, how to install trust in the system and how to facilitate smooth interaction between human and AI. This paper provides an overview of the topics and challenges faced by designers working on higher levels of automation in aviation, with a focus on comparing the identified research gaps and the challenges faced by the practitioners in order to bridge the gap between theory and practice. A comparison with National Academies of Sciences, Engineering, and Medicine's guidance on human-AI teaming reveals areas of agreement and highlights key research directions, while also indicating that current levels of automation are insufficient for effective teaming between human and AI, underscoring the need for further collaboration to address remaining challenges.