Attention for Implicit Discourse Relation Recognition

Attention for Implicit Discourse Relation Recognition Andre Cianflone, Leila Kosseim Department of Engineering and Computer Science Concordia University {a cianfl, kosseim}@encs.concordia.ca Abstract Implicit discourse relation recognition remains a challenging task as state-of-the-art approaches reach F1 scores ranging from 9.95% to 37.67% on the 2016 CoNLL shared task. In our work, we explore the use of a neural network which exploits the strong correlation between pairs of words across two discourse arguments that implicitly signal a discourse relation. We present a novel approach to Implicit Discourse Relation Recognition that uses an encoder-decoder model with attention. Our approach is based on the assumption that a discourse argument is “generated” from a previous argument and conditioned on a latent discourse relation, which we detect. Experiments show that our model achieves an F1 score of 38.25% on fine-grained classification, outperforming previous approaches and performing comparatively with state-of-the-art on coarse-grained classification, while computing alignment parameters without the need for additional pooling and fully connected layers. Keywords: discourse relation recognition, sequence-to-sequence 1. Introduction Shallow discourse relation recognition refers to the auto- matic identification of the relation between two segments of text. For example in: (1) I will go to Scotland after I complete my studies. The underlined discourse connective connects the first dis- course argument (in italic) to the second discourse argu- ment (in bold) via a temporal relation. Connectives consti- tute strong signals to identify discourse relations. In fact, given two arguments and a discourse connective many dis- course parsers at the 2016 CoNLL Shared Task on Multi- lingual Shallow Discourse Parsing (SDP) (Xue et al., 2016) were around 78% accurate in recognizing the discourse re- lation on the SDP blind dataset. On the other hand, in im- plicit relations no connective is used. This is the case in: (2) I need to file my taxes.Tomorrow is the deadline. In (2) the connective because is implied and the contin- gency relation is understood by the context. Unfortunately when the connective is absent, identifying the relation au- tomatically becomes much more challenging. At the same 2016 CoNLL SDP shared task, the best implicit discourse relation (IDR) score on the blind test set without connec- tives reached 37.67% (Xue et al., 2016). In this paper we present a model to automatically recognize implicit dis- course relations using an encoder-decoder with attention, a cross-argument word-pair alignment statistic in this con- text. We show that our model, with an F1 score of 38.25, outperforms other approaches on fine-grained classifica- tion, while performing comparatively with the state-of-the- art on coarse-grained classification. 2. Previous Work Beginning with (Zhang et al., 2015a) and notably in the past year with the CoNLL SDP (Xue et al., 2016), neu- ral network techniques have been used for IDR. Most of these models are based on convolutional neural networks (CNN), inspired by (Zhang et al., 2015a) and other work on sentence classification with CNN (such as (Kim, 2014; Zhang et al., 2015b)). The insight into these many works is that neural networks are better suited at …

Content truncated to 500 words