Disfluency Correction using Unsupervised and Semi-supervised Learning

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 3421–3427 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 3421 Disfluency Correction using Unsupervised and Semi-supervised Learning Nikhil Saini∗1, Drumil Trivedi∗1, Shreya Khare2, Tejas I. Dhamecha2, Preethi Jyothi1, Samarth Bharadwaj2, Pushpak Bhattacharyya1 1Indian Institute of Technology Bombay {nikhilra,drumilt,pjyothi,pb}@cse.iitb.ac.in 2IBM Research - India {skhare34,tidhamecha,samarth.b}@in.ibm.com Abstract Spoken language is different from the written language in its style and structure. Disfluen- cies that appear in transcriptions from speech recognition systems generally hamper the per- formance of downstream NLP tasks. Thus, a disfluency correction system that converts dis- fluent to fluent text is of great value. This pa- per introduces a disfluency correction model that translates disfluent to fluent text by draw- ing inspiration from recent encoder-decoder unsupervised style-transfer models for text. We also show considerable benefits in perfor- mance when utilizing a small sample of 500 parallel disfluent-fluent sentences in a semi- supervised way. Our unsupervised approach achieves a BLEU score of 79.39 on the Switch- board corpus test set, with further improve- ment to a BLEU score of 85.28 with semi- supervision. Both are comparable to two com- petitive fully-supervised models. 1 Introduction Disfluencies are disruptions to the regular flow of speech, typically occurring in conversational speech. They include filler pauses such as uh and um, word repetitions, irregular elongations, dis- course markers, conjunctions, and restarts. For example, the disfluent sentence “well we’re actu- ally uh we’re getting ready” has its fluent form as, “we’re getting ready”. Here, the words highlighted in green, blue and red refer to discourse, filler and restart disfluencies, respectively. Disfluencies in the text can alter its syntactic and semantic structure, thereby adversely affecting the performance of downstream NLP tasks such as in- formation extraction, summarization, translation, and parsing (Charniak and Johnson, 2001; Johnson and Charniak, 2004). These tasks also employ pre- trained language models that are typically trained ∗Joint first authors to expect fluent text. This motivates the need for disfluency correction systems that convert disflu- ent to fluent text. Prior work has predominantly focused on the problem of disfluency detection (Za- yats et al., 2016; Wang et al., 2018; Dong et al., 2019). Inspired by recent work on unsupervised machine translation and style-transfer models for text, we propose an unsupervised encoder-decoder based model to tackle the problem of disfluency cor- rection. Our model does not require access to a par- allel corpus of disfluent and fluent sentences. We also show a semi-supervised variant of our model that uses a small amount of parallel disfluent-fluent text and significantly improves performance. To our knowledge, this is the first work to use state- of-the-art unsupervised models for the task of dis- fluency correction. Our main contributions are as follows: • We cast the problem of disfluency correc- tion as one of translation from disfluent to fluent text and we propose an unsupervised transformer-based encoder-decoder model for disfluency correction. • We compare and contrast an unsupervised and semi-supervised approach for disfluency correction, …

Content truncated to 500 words