A Systematic Study of Neural Discourse Models for Implicit Discourse Relation

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 281–291, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics A Systematic Study of Neural Discourse Models for Implicit Discourse Relation Attapol T. Rutherford Yelp San Francisco, CA, USA teruth@yelp.com Vera Demberg Saarland University Saarbr¨ucken, Germany vera@coli.uni-saarland.de Nianwen Xue Brandeis University Waltham, MA, USA xuen@brandeis.edu Abstract Inferring implicit discourse relations in natural language text is the most difficult subtask in discourse parsing. Many neu- ral network models have been proposed to tackle this problem. However, the com- parison for this task is not unified, so we could hardly draw clear conclusions about the effectiveness of various archi- tectures. Here, we propose neural net- work models that are based on feedfor- ward and long-short term memory archi- tecture and systematically study the effects of varying structures. To our surprise, the best-configured feedforward architecture outperforms LSTM-based model in most cases despite thorough tuning. Further, we compare our best feedforward system with competitive convolutional and recur- rent networks and find that feedforward can actually be more effective. For the first time for this task, we compile and pub- lish outputs from previous neural and non- neural systems to establish the standard for further comparison. 1 Introduction The discourse structure of a natural language text has been analyzed and conceptualized under vari- ous frameworks (Mann and Thompson, 1988; Las- carides and Asher, 2007; Prasad et al., 2008). The Penn Discourse TreeBank (PDTB) and the Chi- nese Discourse Treebank (CDTB), currently the largest corpora annotated with discourse structures in English and Chinese respectively, view the dis- course structure of a text as a set of discourse re- lations (Prasad et al., 2008; Zhou and Xue, 2012). Each discourse relation (e.g. causal or temporal) is grounded by a discourse connective (e.g. because or meanwhile) taking two text segments as argu- ments (Prasad et al., 2008). Implicit discourse re- lations are those where discourse connectives are omitted from the text and yet the discourse rela- tions still hold. While classifying explicit discourse relations is relatively easy, as the discourse connective itself provides a strong cue for the discourse relation (Pitler et al., 2008), the classification of implicit discourse relations has proved to be notoriously hard and remained one of the last missing pieces in an end-to-end discourse parser (Xue et al., 2015). In the absence of explicit discourse connectives, implicit discourse relations have to be inferred from their two arguments. Previous approaches on inferring implicit discourse relations have typ- ically relied on features extracted from their two arguments. These features include the Cartesian products of the word tokens in the two arguments as well as features manually crafted from vari- ous lexicons such as verb classes and sentiment lexicons (Pitler et al., 2009; Rutherford and Xue, 2014). These lexicons are used mainly to offset the data sparsity problem created by pairs of word tokens used directly as features. Neural network models are an attractive alter- native …

Content truncated to 500 words