EMO&LY (EMOtion and AnomaLY) : A new corpus for anomaly detection in an audiovisual stream with emotional context.

EMO&LY (EMOtion and AnomaLY) : A New Corpus for Anomaly Detection in an Audiovisual Stream with Emotional Context C´edric Fayet1,2, Arnaud Delhay1,2, Damien Lolive1,2, Pierre-Franc¸ois Marteau1,3 1IRISA - EXPRESSION Team, Lannion&Vannes, France 2Universit´e de Rennes 1, Rennes, France 3Universit´e de Bretagne Sud, Vannes, France {cedric.fayet, arnaud.delhay, damien.lolive, pierre-francois.marteau}@irisa.fr Abstract This paper presents a new corpus, called EMOLY (EMOtion and AnomaLY), composed of speech and facial video records of subjects that contains controlled anomalies. As far as we know, to study the problem of anomaly detection in discourse by using machine learning classification techniques, no such corpus exists or is available to the community. In EMOLY, each subject is recorded three times in a recording studio, by filming his/her face and recording his/her voice with a HiFi microphone. Anoma- lies in discourse are induced or acted. At this time, about 8,65 hours of usable audiovisual recording on which we have tested classical classification techniques (GMM or One Class-SVM plus threshold classifier) are available. Results confirm the usability of the anomaly induction mechanism to produce anomalies in discourse and also the usability of the corpus to improve detection techniques. Keywords: Audiovisual corpus, Acted emotion, Reading task, Anomaly detection, Unbalanced data 1. Introduction According to (Chandola et al., 2009), ”an anomaly is de- fined as a pattern that does not conform to an expected normal behavior”. Anomaly detection systems have been applied to various domains: intrusion detection (Axelsson, 2000), fraud detection (Abdallah et al., 2016), sensor net- works (Park et al., 2010), flight safety monitoring (Li et al., 2011) or video surveillance (Ko, 2008). In the field of human abnormal behavior detection, the fo- cus is put on tasks like crowd modeling, violence detection (Mehran et al., 2009; Gu et al., 2014), human activity detec- tion (Chaquet et al., 2013). In the corpora used or presented in those works, speech and/or face of the subject are usually not available. On the other hand, speech corpora available in the scope of anomaly detection are focused on disease, stress (Hansen et al., 1997) and detection of depression. For instance, the NKI-CCRT corpus (Clapham et al., 2012) has been built to study speech intelligibility before and after cancer. In (Giraud et al., 2013), a corpus containing mul- timodal expressions of stress during a public speaking task is presented. While this corpus is not dedicated to anomaly detection, it has the advantage to be multimodal, notably including speech and face of the participant. Such a corpus enables to study jointly facial expressions and speech. When it comes to emotions, some corpora are also available like the Belfast Naturalistic Database (Douglas-Cowie et al., 2003) containing records of people discussing emotive subjects or the EmoTV1 corpus (Abrilian et al., 2005) con- taining TV interviews bearing naturalistic non-acted emo- tionally marked data. All those corpora have been designed and collected to an- swer to specific scientific questions. As far as we know, no emotional audiovisual corpus exists containing con- trolled, acted or natural anomalies to address specifically the anomaly detection …

Content truncated to 500 words