A Semi-Supervised Approach to Improve Classification of Infrequent Discourse Relations Using Feature Vector Extension

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 399–409, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics A Semi-Supervised Approach to Improve Classification of Infrequent Discourse Relations using Feature Vector Extension Hugo Hernault hugo@mi.ci.i. u-tokyo.ac.jp Danushka Bollegala danushka@iba.t. u-tokyo.ac.jp Graduate School of Information Science & Technology The University of Tokyo 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, Japan Mitsuru Ishizuka ishizuka@i. u-tokyo.ac.jp Abstract Several recent discourse parsers have em- ployed fully-supervised machine learning ap- proaches. These methods require human an- notators to beforehand create an extensive training corpus, which is a time-consuming and costly process. On the other hand, un- labeled data is abundant and cheap to col- lect. In this paper, we propose a novel semi-supervised method for discourse rela- tion classification based on the analysis of co- occurring features in unlabeled data, which is then taken into account for extending the fea- ture vectors given to a classifier. Our exper- imental results on the RST Discourse Tree- bank corpus and Penn Discourse Treebank in- dicate that the proposed method brings a sig- nificant improvement in classification accu- racy and macro-average F-score when small training datasets are used. For instance, with training sets of c.a. 1000 labeled instances, the proposed method brings improvements in ac- curacy and macro-average F-score up to 50% compared to a baseline classifier. We believe that the proposed method is a first step towards detecting low-occurrence relations, which is useful for domains with a lack of annotated data. 1 Introduction Automatic detection of discourse relations in natu- ral language text is important for numerous tasks in NLP, such as sentiment analysis (Somasundaran et al., 2009), text summarization (Marcu, 2000) and di- alogue generation (Piwek et al., 2007). However, most of the recent work employing discourse re- lation classifiers are based on fully-supervised ma- chine learning approaches (duVerle and Prendinger, 2009; Pitler et al., 2009; Lin et al., 2009). Two of the main corpora with discourse annotations are the RST Discourse Treebank (RSTDT) (Carlson et al., 2001) and the Penn Discourse Treebank (PDTB) (Prasad et al., 2008a), which are both based on the Wall Street Journal (WSJ) corpus. In the RSTDT, annotation is done using 78 fine-grained discourse relations, which are usually grouped into 18 coarser-grained relations. Each of these relations has furthermore several possible con- figurations for its arguments—its ‘nuclearity’ (Mann and Thompson, 1988). In practice, a classifier trained on these coarse-grained relations must solve a 41-class classification problem. Some of the re- lations corresponding to these classes are relatively more frequent in the corpus, such as the ELAB- ORATION[N][S] relation (4441 instances), or the ATTRIBUTION[S][N] relation (1612 instances).1 However, other relation types occur very rarely, such as TOPIC-COMMENT[S][N] (2 instances), or EVALUATION[N][N] (3 instances). A similar phe- nomenon can be observed in PDTB, in which 15 level-two relations are employed: Some, such as EXPANSION.CONJUNCTION, occur as often as 8759 times throughout the corpus, whereas the remainder of the relations, such as EXPANSION.EXCEPTION and COMPARISON.PRAGMATIC CONCESSION, can appear as …

Content truncated to 500 words