Joint, Incremental Disfluency Detection and Utterance Segmentation from Speech

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 326–336, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Joint, Incremental Disfluency Detection and Utterance Segmentation from Speech Julian Hough and David Schlangen Dialogue Systems Group // CITEC // Faculty of Linguistics and Literature Bielefeld University firstname.lastname@uni-bielefeld.de Abstract We present the joint task of incremen- tal disfluency detection and utterance seg- mentation and a simple deep learning sys- tem which performs it on transcripts and ASR results. We show how the constraints of the two tasks interact. Our joint-task system outperforms the equivalent indi- vidual task systems, provides competitive results and is suitable for future use in con- versation agents in the psychiatric domain. 1 Introduction Artificial conversational systems promise to be a valuable addition to the existing set of psychi- atric health care delivery solutions. As artificial systems, they can ensure that interview protocols are followed, and, perhaps surprisingly, due to being “just a computer”, even seem to increase their interlocutors’ willingness to disclose (Lucas et al., 2014). Interactions with such conversational agents have been shown to contain interpretable markers of psychological distress, such as rate of filled pauses, speaking rate, and various tempo- ral, utterance and turn-related interactional fea- tures (DeVault et al., 2013). Filled pauses and dis- fluencies in general have also been shown to pre- dict outcomes to psychiatric treatment (Howes et al., 2012; McCabe et al., 2013). Currently, these systems are only used to elicit material that is then analysed offline. For offline analysis of transcripts with gold standard utter- ance segmentation, much work exists on detecting disfluencies (Johnson and Charniak, 2004; Qian and Liu, 2013; Honnibal and Johnson, 2014). To enable more cost-effective analysis, however, and possibly even let the interaction script itself be de- pendent on an analysis hypothesis, it would be bet- ter to be able to work directly off the speech sig- nal, and online (incrementally). This is what we explore in this paper, presenting and evaluating a model that works with online, incremental speech recognition output to detect disfluencies with var- ious degrees of fine-grainedness. As a second contribution, we combine incre- mental disfluency detection with another lower- level task that is important for responsive con- versational systems, namely the detection of turn- taking opportunities through detection of utter- ance boundaries. (See for example (Schlangen and Skantze, 2011) for arguments for incremen- tal processing and responsive turn-taking in con- versational systems, and (Schlangen, 2006; At- terer et al., 2008; Raux, 2008; Manuvinakurike et al., 2016, inter alia) for examples of incremen- tal utterance segmentation). Besides both being relevant for interactive health assessment systems, these tasks also have an immanent connection, as the approach typically used for turn-end detection is simply waiting for a silence of a certain dura- tion, and hence is mislead by intra-turn silent dis- fluencies. Similarly, without gold standard seg- mentation, disfluent restarts and repairs may be predicted at fluent utterance boundaries. We hence conjecture …

Content truncated to 500 words