The importance of fillers for text representations of speech transcripts

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7985–7993, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7985 The importance of fillers for text representations of speech transcripts Tanvi Dinkar1∗, Pierre Colombo1,2∗, Matthieu Labeau1, Chlo´e Clavel1 1LTCI, Telecom Paris, Institut Polytechnique de Paris, 2IBM GBS France 1firstname.lastname@telecom-paris.fr Abstract While being an essential component of spoken language, fillers (e.g. “um” or “uh”) often remain overlooked in Spoken Language Understanding (SLU) tasks. We explore the possibility of representing them with deep contex- tualised embeddings, showing improve- ments on modelling spoken language and two downstream tasks — predict- ing a speaker’s stance and expressed confidence. 1 Introduction Disfluencies are interruptions in the regular flow of speech, such as pausing silently, repeat- ing words, or interrupting oneself to correct something said previously (Fraundorf et al., 2018). They commonly occur in spoken lan- guage, as spoken language is rarely fluent. Fillers are a type of disfluency that can be a sound (“um” or “uh”) filling a pause in an utterance or conversation. Recent work has shown that contextualised embeddings pre-trained on large written cor- pora can be fine-tuned on smaller spoken lan- guage corpora to learn structures of spoken language (Tran et al., 2019). However, for NLP tasks, fillers and all disfluencies are typi- cally removed in pre-processing, as NLP mod- els achieve highest accuracy on syntactically correct utterances. This contradicts linguistic studies, which show that fillers are an essen- tial and informative part of spoken language (Clark and Fox Tree, 2002; Yoshida and Lick- ley, 2010; Brennan and Williams, 1995; Corley et al., 2007; Stolcke and Shriberg, 1996). ∗Equal contribution So far, the information carried by fillers has only been studied using hand crafted fea- tures, for example in Le Grezause (2017); Saini (2017); Dinkar et al. (2020). Besides, Barriere et al. (2017) show that pre-trained word em- beddings such as Word2vec (Mikolov et al., 2013), have poor representation of spontaneous speech words such as “uh”, as they are trained on written text and do not carry the same meaning as when used in speech. We address the matter of representing fillers with deep con- textualised word representations (Devlin et al., 2019), and investigate their usefulness in NLP tasks for spoken language, without handcraft- ing features. Hence, the present work is motivated by the following observations: (1) Fillers play an im- portant role in spoken language. For example, a speaker can use fillers to inform the listener about the linguistic structure of their utter- ance, such as in their (difficulties of) selection of appropriate vocabulary while informing the listener about a pause in their upcoming speech stream (Clark and Fox Tree, 2002). (2) Fillers and prosodic cues have also been linked to a speaker’s Feeling of Knowing (FOK) or ex- pressed confidence, that is, a speaker’s certainty or commitment to a statement (Smith and Clark, 1993). Brennan and Williams (1995) ob- served that fillers and prosodic cues contribute to the listener’s perception of the speaker’s ex- pressed confidence in …

Content truncated to 500 words