DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Recognizing Frustration of Drivers from Face Video Recordings and Brain Activation Measurements with Functional Near Infrared Spectroscopy

Ihme, Klas and Unni, Anirudh and Zhang, Meng and Rieger, Jochem and Jipp, Meike (2018) Recognizing Frustration of Drivers from Face Video Recordings and Brain Activation Measurements with Functional Near Infrared Spectroscopy. Frontiers in Human Neuroscience, 12, p. 327. Frontiers Media S.A.. doi: 10.3389/fnhum.2018.00327. ISSN 1662-5161.

[img] PDF

Official URL: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00327/full


Experiencing frustration while driving can harm cognitive processing, result in aggressive behavior, and hence negatively influence driving performance and traffic safety. Being able to automatically detect frustration would allow adaptive driver assistance and automation systems to adequately react to a driver's frustration and mitigate potential negative consequences. To identify reliable and valid indicators of drivers' frustration, we conducted two driving simulator experiments. In the first experiment, we aimed to reveal facial expressions that indicate frustration in continuous video recordings of the driver's face taken while driving highly realistic simulator scenarios in which frustrated or non-frustrated emotional states were experienced. An automated analysis of facial expressions combined with multivariate logistic regression classification revealed that frustrated time intervals can be discriminated from non-frustrated ones with accuracy of 62.0 % (mean over 30 participants). A further analysis of the facial expressions revealed that frustrated drivers tend to activate muscles in the mouth region (chin raiser, lip pucker, lip pressor). In the second experiment, we measured cortical activation with whole head functional near-infrared spectroscopy (fNIRS) while participants experienced frustrating and non-frustrating driving simulator scenarios. Multivariate logistic regression applied to the fNIRS measurements allowed us to discriminate between frustrated and non-frustrated driving intervals with higher accuracy of 78.1 % (mean over 12 subjects). Frustrated driving intervals were indicated by increased activation in the inferior frontal, putative premotor, and occipito-temporal cortices. Our results show that facial and cortical markers of frustration can be informative for time resolved driver state identification in complex realistic driving situations. The markers derived here can potentially be used as an input for future adaptive driver assistance and automation systems that detect driver frustration and adaptively react to mitigate it.

Item URL in elib:https://elib.dlr.de/117957/
Document Type:Article
Title:Recognizing Frustration of Drivers from Face Video Recordings and Brain Activation Measurements with Functional Near Infrared Spectroscopy
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Ihme, KlasUNSPECIFIEDhttps://orcid.org/0000-0002-7911-3512UNSPECIFIED
Unni, AnirudhUniversität OldenburgUNSPECIFIEDUNSPECIFIED
Rieger, JochemUniversität OldenburgUNSPECIFIEDUNSPECIFIED
Date:August 2018
Journal or Publication Title:Frontiers in Human Neuroscience
Refereed publication:Yes
Open Access:Yes
Gold Open Access:Yes
In ISI Web of Science:Yes
Page Range:p. 327
Publisher:Frontiers Media S.A.
Keywords:Frustration, Driver State Recognition, Facial Expressions, functional Near Infrared Spectroscopy, Adaptive Automation
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Transport
HGF - Program Themes:Terrestrial Vehicles (old)
DLR - Research area:Transport
DLR - Program:V BF - Bodengebundene Fahrzeuge
DLR - Research theme (Project):V - Fahrzeugintelligenz (old)
Location: Braunschweig
Institutes and Institutions:Institute of Transportation Systems > Human Factors
Deposited By: Ihme, Klas
Deposited On:06 Sep 2018 15:11
Last Modified:02 Nov 2023 09:55

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.