Comparing Word Representations for Implicit Discourse Relation Classification

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2201–2211, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Comparing Word Representations for Implicit Discourse Relation Classification Chlo´e Braud ALPAGE, Univ Paris Diderot & INRIA Paris-Rocquencourt 75013 Paris - France chloe.braud@inria.fr Pascal Denis MAGNET, INRIA Lille Nord-Europe 59650 Villeneuve d’Ascq - France pascal.denis@inria.fr Abstract This paper presents a detailed compar- ative framework for assessing the use- fulness of unsupervised word representa- tions for identifying so-called implicit dis- course relations. Specifically, we compare standard one-hot word pair representations against low-dimensional ones based on Brown clusters and word embeddings. We also consider various word vector combi- nation schemes for deriving discourse seg- ment representations from word vectors, and compare representations based either on all words or limited to head words. Our main finding is that denser represen- tations systematically outperform sparser ones and give state-of-the-art performance or above without the need for additional hand-crafted features. 1 Introduction Identifying discourse relations is an important task, either to build a discourse parser or to help other NLP systems such as text summarization or question-answering. This task is relatively straightforward when a discourse connective, such as but or because, is used (Pitler and Nenkova, 2009). The identification becomes much more challenging when such an overt marker is lacking, and the relation needs to be inferred through other means. In (1), the presence of the pair of verbs (rose,tumbled) triggers a Contrast relation. Such relations are extremely pervasive in real text cor- pora: they account for about 50% of all relations in the Penn Discourse Treebank (Prasad et al., 2008). (1) [ Quarterly revenue rose 4.5%, to $2.3 billion from $2.2 billion]arg1 [ For the year, net in- come tumbled 61% to $86 million, or $1.55 a share]arg2 Automatically classifying implicit relations is dif- ficult in large part because it relies on numerous factors, ranging from syntax, and tense and as- pect, to lexical semantics and even world knowl- edge (Asher and Lascarides, 2003). Consequently, a lot of previous work on this problem have at- tempted to incorporate some of these information into their systems. These assume the existence of syntactic parsers and lexical databases of var- ious kinds, which are available but for a few lan- guages, and they often involve heavy feature en- gineering (Pitler et al., 2009; Park and Cardie, 2012). While acknowledging this knowledge bot- tleneck, this paper focuses on trying to predict im- plicit relations based on easily accessible lexical features, targeting in particular simple word-based features, such as pairs like (rose,tumbled) in (1). Most previous studies on implicit relations, go- ing back to (Marcu and Echihabi, 2002), in- corporate word-based information in the form of word pair features defined across the pair of text segments to be related. Such word pairs are often encoded in a one-hot represen- tation, in which each possible word pair corre- sponds to a single component of a very high- dimensional vector. From a machine learning perspective, this type of …

Content truncated to 500 words