Learning Connective-based Word Representations for Implicit Discourse Relation Identification

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 203–213, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Learning Connective-based Word Representations for Implicit Discourse Relation Identification Chlo´e Braud CoAStaL, Dep of Computer Science University of Copenhagen University Park 5, 2100 Copenhagen, Denmark braud@di.ku.dk Pascal Denis Magnet Team, INRIA Lille – Nord Europe 59650 Villeneuve dAscq, France pascal.denis@inria.fr Abstract We introduce a simple semi-supervised ap- proach to improve implicit discourse relation identification. This approach harnesses large amounts of automatically extracted discourse connectives along with their arguments to con- struct new distributional word representations. Specifically, we represent words in the space of discourse connectives as a way to directly encode their rhetorical function. Experiments on the Penn Discourse Treebank demonstrate the effectiveness of these task-tailored repre- sentations in predicting implicit discourse re- lations. Our results indeed show that, despite their simplicity, these connective-based rep- resentations outperform various off-the-shelf word embeddings, and achieve state-of-the-art performance on this problem. 1 Introduction A natural distinction is often made between ex- plicit and implicit discourse relations depending on whether they are lexicalized by a connective or not, respectively. To illustrate, the Contrast relation in example (1a) is triggered by the connective but, while it is not overtly marked in example (1b).1 Given the lack of strong explicit cues, the identi- fication of implicit relations is a much more chal- lenging and still open problem. The typically low performance scores for this task also hinder the de- velopment of text-level discourse parsers (Lin et al., 2010; Xue et al., 2015): implicit discourse relations 1These examples are taken from documents wsj 0008 and wsj 0037, respectively, of the PDTB. account for around half of the data for different gen- res and languages (Prasad et al., 2008; Sporleder and Lascarides, 2008; Taboada, 2006; Subba and Di Eu- genio, 2009; Soria and Ferrari, 1998; Versley and Gastel, 2013). (1) a. The house has voted to raise the ceiling to $3.1 trillion, but the Senate isn’t expected to act until next week at the earliest. b. That’s not to say that the nutty plot of “A Wild Sheep Chase” is rooted in reality. It’s imaginative and often funny. The difficulty of this task lies in its dependence on a wide variety of linguistic factors, ranging from syntax, lexical semantics and also world knowl- edge (Asher and Lascarides, 2003). In order to deal with this issue, a common approach is to exploit hand-crafted resources to design features captur- ing lexical, temporal, modal, or syntactic informa- tion (Pitler et al., 2009; Park and Cardie, 2012). By contrast, more recent work show that using simple low-dimensional word-based representations, either cluster-based or distributed (aka word embeddings), yield comparable or better performance (Rutherford and Xue, 2014; Braud and Denis, 2015), while dis- pensing with feature engineering. While standard low-dimensional word represen- tations appear to encode relevant linguistic infor- mation, they have not been built with the specific rhetorical task in mind. A natural question is there- fore whether one …

Content truncated to 500 words