DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Graph Convolutional Networks for Frustration Recognition of Drivers

Le Houcq Corbi, Raquel (2021) Graph Convolutional Networks for Frustration Recognition of Drivers. Master's, Technische Universität München.

[img] PDF - Only accessible within DLR


High frustration levels during driving are a critical problem for road safety during manual driving and play an important role in the user experience on all automation levels. For this reason, detecting frustration is essential to build more user-focused systems that adapt their behaviour to the user's needs. In this thesis we propose a fully automated method to detect frustration levels of drivers. The method was trained with a unique and labeled dataset containing real life driving situations, collected in a driving simulator with 50 participants. The video recordings were labeled with a post-hoc continuous frustration rating. The first step of our method was to extract facial landmarks from the video recordings to build a graph, representing facial structures at each frame. The graph was then used to train state-of-the-art Spatio-Temporal Graph Convolutional Networks with residual links between layers. The network is formed of three input branches, containing landmark, velocity, and edge features, each processed first separately and then concatenated into one main network stream. The model was additionally trained combining the information from high frequency and low frequency data, using a two-path network, to capture both fast movements and the semantics of the data. The results show promising results for the real-time recognition of frustration levels, with an accuracy of 79.4±7.4 % when allowing a prediction difference of 0.1. The model presents a new method to get objective frustration ratings from video feeds of drivers by using subjective labels of several drivers to train our network.

Item URL in elib:https://elib.dlr.de/144556/
Document Type:Thesis (Master's)
Title:Graph Convolutional Networks for Frustration Recognition of Drivers
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Date:October 2021
Refereed publication:No
Open Access:No
Number of Pages:76
Keywords:affective computing, frustration recognition, graph convolutional networks, driver emotions
Institution:Technische Universität München
Department:Institute for Human-Machine Communication
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Transport
HGF - Program Themes:Road Transport
DLR - Research area:Transport
DLR - Program:V ST Straßenverkehr
DLR - Research theme (Project):V - NGC KoFiF (old)
Location: Braunschweig
Institutes and Institutions:Institute of Transportation Systems > Information Flow Modelling in Mobility Systems, BS
Deposited By: Bosch, Esther Johanna
Deposited On:07 Dec 2021 11:58
Last Modified:07 Dec 2021 11:58

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.