Speak to your Software Visualization - Exploring Component-Based Software Architectures in Augmented Reality with a Conversational Interface

Abstract

Exploring of software architectures with software visualization in Augmented Reality (AR) is possible with different interaction methods, such gesture, gaze, and speech. For interaction with speech (i.e., natural language), we present an architecture and an implementation of conversational interfaces for the Microsoft HoloLens device. We aim to remedy some peculiarities of AR devices, but also enhancing the exploration task at hand. To implement the conversational interface different natural language processing (NLP) components such as natural language generation and intent recognition are typically required. Our proposed architecture integrates conversational components with the AR-based software visualization. We describe its implementation based on different user utterances, where the system provides information about the to-be-explored component-based software architecture in the form of adjusted visualizations and speech-based results. We apply out tool to explore OSGi-based software architectures.