Closing the Gap: Domain Adaptation from Explicit to Implicit Discourse Relations

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2219–2224, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Closing the Gap: Domain Adaptation from Explicit to Implicit Discourse Relations Yangfeng Ji Gongbo Zhang Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology {jiyfeng,gzhang64,jacobe}@gatech.edu Abstract Many discourse relations are explicitly marked with discourse connectives, and these examples could potentially serve as a plentiful source of training data for recog- nizing implicit discourse relations. How- ever, there are important linguistic differ- ences between explicit and implicit dis- course relations, which limit the accuracy of such an approach. We account for these differences by applying techniques from domain adaptation, treating implic- itly and explicitly-marked discourse rela- tions as separate domains. The distribu- tion of surface features varies across these two domains, so we apply a marginalized denoising autoencoder to induce a dense, domain-general representation. The label distribution is also domain-specific, so we apply a resampling technique that is simi- lar to instance weighting. In combination with a set of automatically-labeled data, these improvements eliminate more than 80% of the transfer loss incurred by train- ing an implicit discourse relation classifier on explicitly-marked discourse relations. 1 Introduction Discourse relations reveal the structural orga- nization of text, potentially supporting applica- tions such as summarization (Louis et al., 2010; Yoshida et al., 2014), sentiment analysis (Soma- sundaran et al., 2009), and coherence evaluation (Lin et al., 2011). While some relations are sig- naled explicitly with connectives such as how- ever (Pitler et al., 2008), many more are im- plicit. Expert-annotated datasets of implicit dis- course relations are expensive to produce, so it would be preferable to use weak supervision, by automatically labeling instances with explicit con- nectives (Marcu and Echihabi, 2003). However, Sporleder and Lascarides (2008) show that models trained on explicitly marked ex- amples generalize poorly to implicit relation iden- tification. They argued that explicit and implicit examples may be linguistically dissimilar, as writ- ers tend to avoid discourse connectives if the dis- course relation could be inferred from context (Grice, 1975). Similar observations are made by Rutherford and Xue (2015), who attempt to add automatically-labeled instances to improve super- vised classification of implicit discourse relations. In this paper, we approach this problem from the perspective of domain adaptation. Specifically, we argue that the reason that automatically-labeled examples generalize poorly is due to domain mis- match from the explicit relations (source domain) to the implicit relations (target domain). We pro- pose to close the gap by using two simple meth- ods from domain adaptation: (1) feature represen- tation learning: mapping the source domain and target domain to a shared latent feature space; (2) resampling: modifying the relation distribution in the explicit relations to match the distribution over implicit relations. Our results on the Penn Dis- course Treebank (Prasad et al., 2008) show that these two methods improve the performance on unsupervised discourse relation identification by more than 8.4% on average F1 score across all relation types, an 82% …

Content truncated to 500 words