Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 3693–3703 August 1–6, 2021. ©2021 Association for Computational Linguistics 3693 Best of Both Worlds: Making High Accuracy Non-incremental Transformer-based Disfluency Detection Incremental Morteza Rohanian and Julian Hough Cognitive Science Group School of Electronic Engineering and Computer Science Queen Mary University of London {m.rohanian, j.hough} @qmul.ac.uk Abstract While Transformer-based text classifiers pre- trained on large volumes of text have yielded significant improvements on a wide range of computational linguistics tasks, their imple- mentations have been unsuitable for live in- cremental processing thus far, operating only on the level of complete sentence inputs. We address the challenge of introducing meth- ods for word-by-word left-to-right incremen- tal processing to Transformers such as BERT, models without an intrinsic sense of linear order. We modify the training method and live decoding of non-incremental models to de- tect speech disfluencies with minimum latency and without pre-segmentation of dialogue acts. We experiment with several decoding meth- ods to predict the rightward context of the word currently being processed using a GPT-2 language model and apply a BERT-based dis- fluency detector to sequences, including pre- dicted words. We show our method of incre- mentalising Transformers maintains most of their high non-incremental performance while operating strictly incrementally. We also evalu- ate our models’ incremental performance to es- tablish the trade-off between incremental per- formance and final performance, using differ- ent prediction strategies. We apply our sys- tem to incremental speech recognition results as they arrive into a live system and achieve state-of-the-art results in this setting. 1 Introduction Conversational systems provide a significant ad- dition to the present approaches in mental health care delivery. Interactions with these conversa- tional agents have been shown to contain observ- able indicators of cognitive states, such as the rate of filled pauses and different temporal and turn- related features (Gratch et al., 2014). Alzheimer’s Disease (AD) patients, for example, have trouble performing tasks that leverage semantic informa- tion; they have difficulties with verbal fluency and object recognition. AD patients speak more slowly with long pauses and spend extra time looking for the correct word, which leads to speech disfluency (L´opez-de Ipi˜na et al., 2013; Nasreen et al., 2021). Disfluency markers can be key features for identi- fying certain cognitive disorders for application in conversational agents (Rohanian et al., 2020). Such conversational systems are primarily used for content processing, which is then analyzed of- fline. There is much work on detecting disfluen- cies for offline analysis of transcripts. However, given that these disfluency detection models do not work for live systems and depend on rich transcrip- tion data, including pre-segmentation of dialogue acts, to facilitate more cost-effective analysis of other data, we need systems capable of performing directly and incrementally off the speech signal, or at least from the results of automatic speech recognition (ASR) as they arrive in the system. As it receives word-by-word data, an incremen- tal model must …

Content truncated to 500 words