TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2793–2803, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 2793 TED-CDB: A Large-Scale Chinese Discourse Relation Dataset on TED Talks Wanqiu Long ‡†, Bonnie Webber†, and Deyi Xiong‡ † University of Edinburgh, Edinburgh, UK ‡ College of Intelligence and Computing, Tianjin University, Tianjin, China Wanqiu.long@ed.ac.uk, bonnie.webber@ed.ac.uk, dyxiong@tju.edu.cn Abstract As different genres are known to differ in their communicative properties and as previ- ously, for Chinese, discourse relations have only been annotated over news text, we have created the TED-CDB dataset. TED-CDB comprises a large set of TED talks in Chi- nese that have been manually annotated ac- cording to the goals and principles of Penn Dis- course Treebank, but adapted to features that are not present in English. It serves as a unique Chinese corpus of spoken discourse. Bench- mark experiments show that TED-CDB poses a challenge for state-of-the-art discourse rela- tion classifiers, whose F1 performance on 4- way classification is <60%. This is a dramatic drop of 35% from performance on the news text in the Chinese Discourse Treebank. Trans- fer learning experiments have been carried out with the TED-CDB for both same-language cross-domain transfer and same-domain cross- language transfer. Both demonstrate that the TED-CDB can improve the performance of systems being developed for languages other than Chinese and would be helpful for insuffi- cient or unbalanced data in other corpora. The dataset and our Chinese annotation guidelines has been made freely available.1 1 Introduction Recent years have witnessed increasing attention to the properties of discourse for a wide variety of natural language processing (NLP) tasks, e.g., ma- chine translation (Ohtani et al., 2019; Voita et al., 2019), summarization (Isonuma et al., 2019; Xu et al., 2020), machine reading comprehension (Mi- haylov and Frank, 2019). One of those interesting properties is the coherence between clauses and sentences arising from shallow discourse relations. As empirical approaches for modeling discourse relations usually require corpora annotated with 1https://github.com/wanqiulong0923/TED-CDB such relations, Penn Discourse Treebank (PDTB) (Prasad et al., 2008b), based on the idea that the dis- course relations are grounded in an identifiable set of discourse connectives or Altlex expressions, has been widely applied in the field of natural language processing. Largely because PDTB is effective to extract discourse semantic features, it serves as a useful substrate for the development and evaluation of neural models in many downstream NLP applica- tions (Qin et al., 2017; Nie et al., 2019; Narasimhan and Barzilay, 2015). Because for Chinese, discourse relations have only been annotated over news text and few of the resulting corpora are freely available, we have cre- ated the TED-CDB dataset. TED-CDB currently comprises 72 TED talks in Chinese (∼268.1K words), annotated with 15,540 discourse relations — almost 3 times as many as the CDTB (Zhou and Xue, 2015). Because Tonelli et al. (2010) have shown that discourse relations in spoken discourse are expressed differently than in written text, for scenarios involving Chinese spoken discourse (e.g., dialogue, spoken language translation), …

Content truncated to 500 words