Semi-Supervised Disfluency Detection

Proceedings of the 27th International Conference on Computational Linguistics, pages 3529–3538 Santa Fe, New Mexico, USA, August 20-26, 2018. 3529 Semi-Supervised Disfluency Detection Feng Wang1,2, Wei Chen1∗, Zhen Yang1,2 , Qianqian Dong1,2, Shuang Xu1, and Bo Xu1 1Institute of Automation, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China {feng.wang, wei.chen.media,yangzhen2014 dongqianqian2016 ,shuang.xu, xubo}@ia.ac.cn Abstract While the disfluency detection has achieved notable success in the past years, it still severely suffers from the data scarcity. To tackle this problem, we propose a novel semi-supervised ap- proach which can utilize large amounts of unlabelled data. In this work, a light-weight neural net is proposed to extract the hidden features based solely on self-attention without any Recurrent Neural Network (RNN) or Convolutional Neural Network (CNN). In addition, we use the unla- belled corpus to enhance the performance. Besides, the Generative Adversarial Network (GAN) training is applied to enforce the similar distribution between the labelled and unlabelled data. The experimental results show that our approach achieves significant improvements over strong baselines. 1 Introduction A characteristic of spontaneous speech is different from written text, since it’s usually accompanied by disfluencies. Identifying and removing these non-fluent factors would help to improve the spontaneous speech quality. It often plays a significant role in understanding the semantics of these sentences and it’s vital for the downstream NLP tasks, such as question answering, machine translation, and information extraction. I want to flight [ to Boston + { um } + to Denver ] RM IM RP Figure 1: Example of disfluency annotation style in Switchboard corpus. Figure 1 shows a standard annotation of disfluency structure (Shriberg, 1994), which includes three types of annotations: RM (reparandum, words that are discarded, or corrected by the following words), RP (repair, the correct words), and IM (interregnum, such as filled pauses, discourse cue words, etc.) The interruption point (+) marks the end of the reparandum and an optional interregnum. Ignoring the interregnum, disfluencies can be further categorized into three types: repetition (when RP is same as RM), repair, and restart (when RP is empty). Table 1 gives a few examples. However, it is usually difficult to identify the reparandum non-fluent factors. The main challenge behind is that this type of structure is flexible, variable in length, able to occur anywhere in a sentence, and in many cases will be nested. Detecting the disfluency can work in arbitrary form. Recently, a number of approaches based on deep neural networks have been proposed to address the problem of disfluency detection under the framework of sequence labeling or sequence to se- quence(Hough and Schlangen, 2015; Zayats et al., 2016; Wang et al., 2016; Wang et al., 2017). For most of the approaches mentioned above, the researchers switch between the RNN and CNN. But the RNN/CNN shows less flexibility than the self-attention layer which has achieved great success in neural machine translation (Shen et al., 2015; Wu et al., 2016; Gehring et al., 2017) and language understanding *Wei Chen is the corresponding author …

Content truncated to 500 words