Towards Interactive Annotation for Hesitation in Conversational Speech

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1526–1532 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1526 Towards Interactive Annotation for Hesitation in Conversational Speech Jane Wottawa1,2, Marie Tahon1, Apolline Marin2, Nicolas Audibert2 1LIUM / Le Mans Université 2LPP, UMR 7018 CNRS - U. Paris 3 / Sorbonne Nouvelle {jane.wottawa, marie.tahon}@univ-lemans.fr apo.marin@bbox.fr, nicolas.audibert@sorbonne-nouvelle.fr Abstract Manual annotation of speech corpora is costly in both human resources and time. Furthermore, recognizing affects in spontaneous, non acted speech presents a challenge for humans and machines. The aim of the present study is to automatize the labeling of hesitant speech as a marker of expressed uncertainty. That is why, the NCCFr-corpus was manually annotated for degree of hesitation on a continuous scale between -3 and 3 and the affective dimensions activation, valence and control. In total, 5834 chunks of the NCCFr-corpus were manually annotated. Acoustic analyses were carried out based on these annotations. Furthermore, regression models were trained in order to allow automatic prediction of hesitation for speech chunks that do not have a manual annotation. Preliminary results show that the number of filled pauses as well as vowel duration increase with the degree of hesitation, and that automatic prediction of the hesitation degree reaches encouraging RMSE results of 1.6. Keywords: Conversational speech, acoustic analyses, affective dimensions, hesitation, regression models. 1. Introduction To better study hesitation in spontaneous conversational speech, large amounts of annotated data is required. One possibility to speed up the annotation process is to help the annotator by giving some label or value suggestions re- trieved automatically (Marinelli et al., 2019). The second way is to reduce the number of segments to annotate by selecting only relevant data (Fallgren et al., 2019). Interac- tive annotation aims at combining both aspects to reach a good quality of annotation. An active learning strategy is usually included in the process in order to quickly reach a good quality of annotation suggestions by soliciting feed- back from users if necessary (Klie et al., 2018). The quality of annotations can be validated using objective and subjec- tive metrics obtained on a test data set, which represents a compromise between human perception and machine con- sistency. For several years, affect in speech has been encoded using discrete categories such as anger, sadness or neutral speech. However, in many recent papers, researchers preferred using affective dimensions. Firstly theorized by Russel for emo- tional faces (Russel, 1997), the communication of affect can be seen as having three major dimensions of connota- tive meaning: arousal (activation), pleasure (valence) and power (dominance or control). In the field of affective com- puting, mainly activation and valence dimensions are used, and their predictions from speech is considered as a regres- sion problem (Wöllmer et al., 2008). In more recent years, most of the neural prediction systems developed for affec- tive computing, used convolutional or recurrent networks to predict activation and valence (Schmitt et al., 2019). The control (or dominance) dimension is also …

Content truncated to 500 words