Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures

Abstract

Executive cognitive functions like working memory determine the success or failure of a wide variety of different cognitive tasks, such as problem solving, navigation, or planning. Estimation of constructs like working memory load or memory capacity from neurophysiological or psychophysiological signals would enable adaptive systems to respond to cognitive states experienced by an operator and trigger responses designed to support task performance (e.g., by simplifying the exercises of a tutor system when the subject is overloaded Gerjets et al., 2014, or by shutting down distractions from the mobile phone). The determination of cognitive states like working memory load is also useful for automated testing/assessment or for usability evaluation. While there exists a large body of research work on neural and physiological correlates of cognitive functions like working memory activity, fewer publications deal with the application of this research with respect to single-trial detection and real-time estimation of cognitive functions in complex, realistic scenarios. Single-trial classifiers based on brain activity measurements such as electroencephalography (EEG, Kothe and Makeig, 2011; Lotte et al., 2018), functional near-infrared spectroscopy (fNIRS, Putze et al., 2014; Herff et al., 2015), physiological signals (Fairclough et al., 2005; Fairclough, 2008), or eye tracking (Putze et al., 2013) have the potential to classify affective (Koelstra et al., 2010; Heger et al., 2014; Mühl et al., 2014) or cognitive states based upon short segments of data. For this purpose, signal processing and machine learning techniques need to be developed and transferred to real-world user interfaces. The goal of this Frontiers Research Topic was to advance the State-of-the-Art in signal-based modeling of cognitive processes. We were especially interested in research toward more complex and realistic study designs, for example collecting data in the wild or investigating the interaction between different cognitive processes or signal modalities. Bringing together many contributions in one format allowed us to look at the state of convergence or diversity regarding concepts, methods, and paradigms. The accepted manuscripts in this research topic cover a large range of aspects of cognition, reflecting the broadness of the field and its many application domains. A dominant challenge in the research topic is the analysis of cognitive workload (or memory load) from neurological signals. This does not come as a surprise because workload is a thoroughly studied construct and workload models can be immediately exploited, e.g., for adaptive human-machine interaction. While all these manuscripts share a joint research interest, they tackle the challenge of workload modeling in different application domains, with different signals, different classification approaches, and different features. Working memory and attentional control represent two recurring themes through the collection of papers in this research theme. The most prominent modality in this research topic is EEG, drawing from both spectral as well as time-domain features. In multiple articles, EEG is complemented by other modalities: Two of them use fNIRS as a different mode to capture neural activity (two others use fNIRS as single modality); two others use eye tracking as a way to capture visual attention and one also uses physiological signals (such as heart rate and breath rate). This shows that researchers today routinely select which (combinations of) signals are most promising for a given task.