Leveraging Synthetic Discourse Data via Multi-task Learning for Implicit Discourse Relation Recognition

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 476–485, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Leveraging Synthetic Discourse Data via Multi-task Learning for Implicit Discourse Relation Recognition Man Lan and Yu Xu Department of Computer Science and Technology East China Normal University Shanghai, P.R.China mlan@cs.ecnu.edu.cn 51101201049@ecnu.cn Zheng-Yu Niu Baidu Inc. Beijing, P.R.China niuzhengyu@baidu.com Abstract To overcome the shortage of labeled data for implicit discourse relation recogni- tion, previous works attempted to auto- matically generate training data by remov- ing explicit discourse connectives from sentences and then built models on these synthetic implicit examples. However, a previous study (Sporleder and Lascarides, 2008) showed that models trained on these synthetic data do not generalize very well to natural (i.e. genuine) implicit discourse data. In this work we revisit this issue and present a multi-task learning based system which can effectively use synthetic data for implicit discourse relation recognition. Results on PDTB data show that under the multi-task learning framework our models with the use of the prediction of explicit discourse connectives as auxiliary learn- ing tasks, can achieve an averaged F1 im- provement of 5.86% over baseline models. 1 Introduction The task of implicit discourse relation recognition is to identify the type of discourse relation (a.k.a. rhetorical relation) hold between two spans of text, where there is no discourse connective (a.k.a. discourse marker, e.g., but, and) in context to ex- plicitly mark their discourse relation (e.g., Con- trast or Explanation). It can be of great benefit to many downstream NLP applications, such as question answering (QA) (Verberne et al., 2007), information extraction (IE) (Cimiano et al., 2005), and machine translation (MT), etc. This task is quite challenging due to two reasons. First, with- out discourse connective in text, the task is quite difficult in itself. Second, implicit discourse rela- tion is quite frequent in text. For example, almost half the sentences in the British National Corpus held implicit discourse relations (Sporleder and Lascarides, 2008). Therefore, the task of implicit discourse relation recognition is the key to im- proving end-to-end discourse parser performance. To overcome the shortage of manually anno- tated training data, (Marcu and Echihabi, 2002) proposed a pattern-based approach to automat- ically generate training data from raw corpora. This line of research was followed by (Sporleder and Lascarides, 2008) and (Blair-Goldensohn, 2007). In these works, sentences containing cer- tain words or phrases (e.g. but, although) were selected out from raw corpora using a pattern- based approach and then these words or phrases were removed from these sentences. Thus the resulting sentences were used as synthetic train- ing examples for implicit discourse relation recog- nition. Since there is ambiguity of a word or phrase serving for discourse connective (i.e., the ambiguity between discourse and non-discourse usage or the ambiguity between two or more dis- course relations if the word or phrase is used as a discourse connective), the synthetic implicit data would contain a lot of noises. Later, with the re- lease of …

Content truncated to 500 words