Facing driver frustration: Towards real-time in-vehicle frustration estimation based on video streams of the face

Abstract

Drivers frequently experience frustration when facing traffic jams, red lights or badly designed in-vehicle interfaces. Frustration can lead to aggressive behaviors and negative influences on user experience. Affect-aware vehicles that recognize the driver’s degree of frustration and, based on this, offer assistance to reduce the frustration or mitigate its negative effects promise remedy. As a prerequisite, this needs a real-time estimation of current degree of frustration. Consequently, here we describe the development of a classifier that can recognize whether a frustrated facial expression was shown based on video streams of the face. For demonstration of its real-time capabilities, a demonstrator of a frustration-aware vehicle including the classifier, the Frust-O-Meter, is presented. The system is integrated into a driving simulator and consists of (1) a webcam, (2) a preprocessing unit, (3) a user model, (4) an adaptation unit and (5) a user interface. In the current version, a happy song is played once a high degree of frustration is detected. The Frust-O-Meter can form the basis for the development of frustration-aware vehicles and is foreseen to be extended to more modalities as well as more user need-oriented adaption strategies in the near future.