Adaptive Unsupervised Self-training for Disfluency Detection

Proceedings of the 29th International Conference on Computational Linguistics, pages 7209–7218 October 12–17, 2022. 7209 Adaptive Unsupervised Self-training for Disfluency Detection Zhongyuan Wang, Yixuan Wang, Shaolei Wang, Wanxiang Che∗ Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China {zywang, yixuanwang, slwang, car}@ir.hit.edu.cn Abstract Supervised methods have achieved remarkable results in disfluency detection. However, in real-world scenarios, human-annotated data is difficult to obtain. Recent works try to han- dle disfluency detection with unsupervised self- training, which can exploit existing large-scale unlabeled data efficiently. However, their self- training-based methods suffer from the prob- lems of selection bias and error accumulation. To tackle these problems, we propose an adap- tive unsupervised self-training method for dis- fluency detection. Specifically, we re-weight the importance of each training example ac- cording to its grammatical feature and predic- tion confidence. Experiments on the Switch- board dataset show that our method improves 2.3 points over the current SOTA unsupervised method. Moreover, our method is competitive with the current SOTA supervised method. 1 Introduction Disfluency is a characteristic of spontaneous speech which is different from written texts. Detect- ing and removing the non-fluent word sequences in spoken language transcripts can improve the tran- scripts’ quality and provide clean inputs for the downstream NLP tasks, such as parsing, machine translation, and summarization (Tree, 1995; Wang et al., 2020b). As shown in Figure 1, a standard annotation of the disfluency structure indicates the RM (reparandum, words that the speaker intends to discard), IM (interregnum, filled pauses, discourse cue words, etc.), RP (repair, the associated repair) (Shriberg, 1994). Most previous works (Zayats et al., 2016; Wu et al., 2015; Lou and Johnson, 2017; Jamshid Lou et al., 2018; Zayats and Ostendorf, 2019) on disflu- ency detection heavily relies on human-annotated corpora, which is difficult and expensive to ob- tain in practice. Some researchers try to alleviate ∗Email corresponding. I want a cup of tea um I mean coffee I want a cup of tea um I mean coffee RM IM RP Disfluency Detection Figure 1: A sentence with disfluencies annotated. RM=Reparandum, IM=Interregnum, RP=Repair. The preceding RM is corrected by the following RP. this issue with, for instance, self-supervised learn- ing (Wang et al., 2020a) and semi-supervised learn- ing techniques (Wang et al., 2018), but they still need a certain amount of human-annotated corpora to perform high performance. Wang et al. (2020b) completely removes the need for human-annotated data. They first perform disfluency detection with the self-training method in an unsupervised manner, which shows promising performance. They lever- age the pseudo data constructed by rules to train a weak disfluency detection model as a teacher. Then, the student model is fine-tuned in an unsupervised manner, only using the pseudo-labels generated by the teacher. Finally, the student model achieves promising performance. Traditional self-training methods utilize the probability estimations to select high-confidence pseudo-labels for re-training, which encounters two problems. (1) Selection Bias. The strategy that selects samples with the high confidence pseudo- labeled samples tends to neglect the hard and po- …

Content truncated to 500 words