Recognizing Behavioral Factors while Driving: A Real-World Multimodal Corpus to Monitor the Driver’s Affective State

Recognizing Behavioral Factors while Driving: A Real-World Multimodal Corpus to Monitor the Driver’s Affective State Alicia Lotz1, Klas Ihme2, Audrey Charnoz3, Pantelis Maroudis4, Ivan Dmitriev5, Andreas Wendemuth1 1 Otto-von-Guericke-Universit¨at Magdeburg, D-39106 Magdeburg, {firstname}.{surname}@ovgu.de 2 German Aerospace Center, D-38108 Braunschweig, klas.ihme@dlr.de 3 ´Ecole polytechnique f´ed´erale de Lausanne, CH-1015 Lausanne, audrey.charnoz@epfl.ch 4 VALEO Comfort & Driving Assistance Systems, F-93012 Bobigny, pantelis.maroudis@valeo.com 5 Institut VEDECOM Du V´ehicle D´ecarbon´e et Communicant et de sa Mobilit´e, F-78000 Versailles, ivan.dmitriev@vedecom.fr Abstract The presented study concentrates on the collection of emotional multimodal real-world in-car audio, video and physiological signal recordings while driving. To do so, three sensor systems were integrated in the car and four relevant emotional states of the driver were defined: neutral, positive, frustrated and anxious. To gather as natural as possible emotional data of the driver, the subjects needed to be unbiased and were therefore kept unaware of the detailed research objective. The emotions were induced using so-called Wizard-of-Oz experiments, where the drivers believed to be interacting with an automated technical system, which in fact was controlled by a human. Additionally, on board interviews while driving were conducted by an instructed psychologist. To evaluate the collected data, questionnaires were filled out by the subjects before, during and after the data collection. These included monitoring of the drivers perceived state of emotion, stress, sleepiness and thermal sensation but also detailed questionnaires on their driving experience, attitude towards technology and big five OCEAN personality traits. Afterwards, the data was annotated by expert labelers. Exemplary results of the evaluation of the experiments are given in the result section of this paper. They indicate that the emotional states were successfully induced and the annotation results are consistent for both performed annotation approaches. Keywords: in-car emotions, multimodal corpus, multimodal interaction, affective computing, natural emotions 1. Introduction While driving in a car, the driver can be affected by vari- ous emotionally challenging situations. They can either be triggered by the current driving situation, e.g. being cut off by another driver, or caused by a personal event, e.g. recei- ving good news. On the one hand, emotions can effect the driving behavior in positive and negative ways. By sensing fear, the driver is able to perceive a situation as a possi- ble risk and adapt his driving towards the situation, while anger may lead to an underestimation of the risk level and therefore may increase the risk of causing an accident (Lu et al., 2013). On the other hand, positive as well as ne- gative emotions can influence the driving performance in a negative way (Pˆecher et al., 2009; Rhodes and Pivik, 2011; Taubman-Ben-Ari, 2012). Pˆecher et al. show that positive emotions (by listening to music) with high intensity lead the participants to drive with risky behavior (e.g. degraded lateral control, sudden speed decreases). In a similar way, Taubman-Ben-Ari shows that positive emotions (with raw intensity) tempt people to drive in a reckless manner. Ho- wever, positive emotions are rarely considered in terms of road safety, as their occurrence is less …

Content truncated to 500 words