Exploring Contextualized Neural Language Models for Temporal Dependency Parsing

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 941–958, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 941 Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness Reina Akama1,2 Sho Yokoi1,2 Jun Suzuki1,2 Kentaro Inui1,2 1Tohoku University 2RIKEN {reina.a, yokoi, jun.suzuki, inui}@ecei.tohoku.ac.jp Abstract Large-scale dialogue datasets have recently be- come available for training neural dialogue agents. However, these datasets have been re- ported to contain a non-negligible number of unacceptable utterance pairs. In this paper, we propose a method for scoring the quality of ut- terance pairs in terms of their connectivity and relatedness. The proposed scoring method is designed based on findings widely shared in the dialogue and linguistics research commu- nities. We demonstrate that it has a relatively good correlation with the human judgment of dialogue quality. Furthermore, the method is applied to filter out potentially unacceptable ut- terance pairs from a large-scale noisy dialogue corpus to ensure its quality. We experimentally confirm that training data filtered by the pro- posed method improves the quality of neural dialogue agents in response generation.1 1 Introduction Some million-scale datasets such as movie scripts and social media posts have become available in recent years for building neural dialogue agents (Li- son and Tiedemann, 2016; Henderson et al., 2019). Such large-scale datasets can be expected to im- prove the performance of dialogue response genera- tion models based on deep neural networks (DNNs) since the combination of DNNs and large-scale training datasets has led to considerable perfor- mance improvement in many sentence generation tasks (Koehn and Knowles, 2017; Sennrich and Zhang, 2019; Adiwardana et al., 2020). In contrast to the quantity of the data, the quality of the data has often been problematic. For exam- ple, OpenSubtitles (Lison and Tiedemann, 2016; Lison et al., 2018), the most widely used large- scale English dialogue corpus, was constructed by 1The code is publicly available at https://github. com/jqk09a/CoRe-dialogue-filtering. 11.4% 14.6% 23.4% 28.4% 22.2% 1: Strongly disagree 2: Disagree 3: Unsure 4: Agree 5: Strongly agree Figure 1: Is the sequence of the two utterances accept- able as a dialogue? Response acceptability scores are given by humans on the English OpenSubtitles corpus. collecting two consecutive lines of movie subti- tles under the simplified assumption that one line of a movie subtitle is one utterance and the next line is the next utterance follow it. Inevitably, this corpus includes unacceptable utterance pairs from the viewpoint of a conversational sequence, e.g., caused by scene switching or flashback. Several previous studies have identified such flaws and re- ported that the corpus is noisy (Vinyals and Le, 2015; Li et al., 2016; Baheti et al., 2018), where noisy refers to unacceptable utterance pairs in this context. Figure 1 shows the result of our exper- imental investigation regarding the acceptability rate of the utterance pairs in the OpenSubtitles cor- pus.2 It can be noticed from the figure that only half of the utterance pairs can be considered accept- able (i.e., were rated with score 5: Strongly agree …

Content truncated to 500 words