Opinions in Interactions : New Annotations of the SEMAINE Database

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 7049–7055 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 7049 Opinions in Interactions : New Annotations of the SEMAINE Database Valentin Barriere1,∗, Slim Essid2, Chlo´e Clavel2 1Joint Research Center, Via E. Fermi, 2749, I - 21027 Ispra, valentn.barriere@ec.europa.eu 2T´el´ecom Paris, 19 place Marguerite Perey F-91120 Palaiseau, France, chloe.clavel@telecom-paristech.fr Abstract In this paper, we present the process we used in order to collect new annotations of opinions over the multimodal corpus SEMAINE composed of dyadic interactions. The dataset had already been annotated continuously in two affective dimensions related to the emotions: Valence and Arousal. We annotated the part of SEMAINE called Solid SAL composed of 79 interactions between a user and an operator playing the role of a virtual agent designed to engage a person in a sustained, emotionally colored conversation. We aligned the audio at the word level using the available high-quality manual transcriptions. The annotated dataset contains 5627 speech turns for a total of 73,944 words, corresponding to 6 hours 20 minutes of dyadic interactions. Each interaction has been labeled by three annotators at the speech turn level following a three-step process. This method allows us to obtain a precise annotation regarding the opinion of a speaker. We obtain thus a dataset dense in opinions, with more than 48% of the annotated speech turns containing at least one opinion. We then propose a new baseline for the detection of opinions in interactions improving slightly a state of the art model with RoBERTa embeddings. The obtained results on the database are promising with a F1-score at 0.72. Keywords: Opinion, Multimodal Machine Learning, Interactions 1. Introduction The way a human expresses affective phenomena such as an opinion, an emotion or a sentiment is very dif- ferent depending on the media he uses. Whether it is a tweet, a phone call or a real-life interaction will definitely impact the manner to exchange information with another person. Certain kinds of phenomena are easier to detect using specific modalities and specific context. For example, there are many more studies on emotion recognition in speech than in text (Schuller et al., 2009), since often we communicate the emotions through para-linguistic like tone, pitch or emphasis. Similarly, the study of opinion phenomena was initially restricted to text (Irsoy and Cardie, 2014; Balahur et al., 2010) which has a clearer syntax, due to the complexity of the annotation of the phenomenon and the volatility of the linguistic structure in oral language. There are numerous corpora for the study of multi- modal affective phenomena whether they appear in hu- man communication without interaction (Zadeh et al., 2018; Morency et al., 2011), with dyadic interactions (Ringeval et al., 2013; Busso et al., 2017) or with multi- party interactions (Poria et al., 2019a; Lotfian and Busso, 2018). Nevertheless, the majority of the avail- able annotations are centered either on the emotions (McKeown et al., 2012) or on the sentiment (Zadeh et al., …

Content truncated to 500 words