Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 827 - 838 July 10-15, 2022 ©2022 Association for Computational Linguistics Teaching BERT to Wait: Balancing Accuracy and Latency for Streaming Disfluency Detection Angelica Chen∗ New York University ac5968@nyu.edu Vicky Zayats Google Research vzayats@google.com Daniel D. Walker Google Research danwalkeriv@google.com Dirk Padfield Google Research padfield@google.com Abstract In modern interactive speech-based systems, speech is consumed and transcribed incre- mentally prior to having disfluencies removed. This post-processing step is crucial for pro- ducing clean transcripts and high performance on downstream tasks (e.g. machine trans- lation). However, most current state-of-the- art NLP models such as the Transformer op- erate non-incrementally, potentially causing unacceptable delays. We propose a stream- ing BERT-based sequence tagging model that, combined with a novel training objective, is capable of detecting disfluencies in real-time while balancing accuracy and latency. This is accomplished by training the model to decide whether to immediately output a prediction for the current input or to wait for further context. Essentially, the model learns to dynamically size its lookahead window. Our results demon- strate that our model produces comparably ac- curate predictions and does so sooner than our baselines, with lower flicker. Furthermore, the model attains state-of-the-art latency and sta- bility scores when compared with recent work on incremental disfluency detection. 1 Introduction Many modern Natural Language Understanding (NLU) applications (e.g. transcribers, digital voice assistants, and chatbots) use streaming Automatic Speech Recognition (ASR) systems that incremen- tally consume speech, offering real-time transcrip- tion and predictions with minimal delay. How- ever, these systems are often challenged by the presence of disfluencies, which are unintentional speech disruptions such as “um”, “no I meant”, and “I I I I think,” that occur naturally in spontaneous speech. Disfluencies not only hurt the readability of ASR transcripts, but also erode model perfor- mance on downstream tasks, such as machine trans- lation (Hassan et al., 2014) and question answering ∗Work completed as part of the Student Researcher pro- gram at Google. (Gupta et al., 2021). Indeed, even state-of-the-art models such as BERT (Devlin et al., 2019) and T5 (Raffel et al., 2020) exhibit significant drops in performance (as much as 28 and 20 F1 points, re- spectively) on the SQuAD-v2 question-answering benchmark (Rajpurkar et al., 2018) when disfluen- cies are inserted into the questions (Gupta et al., 2021). Past work has shown that a prohibitively large amount of data is needed to train an end-to- end dialogue model that is robust to the presence of disfluencies (Shalyminov et al., 2017). As a result, modern ASR pipelines typically contain a sepa- rate post-processing step that detects and removes disfluencies from the transcript, which has been shown to perform better than end-to-end ASR mod- els that generate fluent text from disfluent speech (Jamshid Lou and Johnson, 2020). Shriberg et al. (1997) introduced the following disfluency schema components that are widely used in disfluency detection research: the reparandum (spoken segment intended to be removed), the …

Content truncated to 500 words