Variational Neural Discourse Relation Recognizer

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 382–391, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Variational Neural Discourse Relation Recognizer Biao Zhang1, Deyi Xiong2∗, Jinsong Su1, Qun Liu3,4, Rongrong Ji1, Hong Duan1, Min Zhang2 Xiamen University, Xiamen, China 3610051 Provincial Key Laboratory for Computer Information Processing Technology Soochow University, Suzhou, China 2150062 ADAPT Centre, School of Computing, Dublin City University3 Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences4 zb@stu.xmu.edu.cn, {jssu, rrji, hduan}@xmu.edu.cn qun.liu@dcu.ie, {dyxiong, minzhang}@suda.edu.cn Abstract Implicit discourse relation recognition is a crucial component for automatic discourse- level analysis and nature language understand- ing. Previous studies exploit discriminative models that are built on either powerful man- ual features or deep discourse representations. In this paper, instead, we explore generative models and propose a variational neural dis- course relation recognizer. We refer to this model as VarNDRR. VarNDRR establishes a directed probabilistic model with a latent con- tinuous variable that generates both a dis- course and the relation between the two ar- guments of the discourse. In order to per- form efficient inference and learning, we in- troduce neural discourse relation models to approximate the prior and posterior distribu- tions of the latent variable, and employ these approximated distributions to optimize a repa- rameterized variational lower bound. This al- lows VarNDRR to be trained with standard stochastic gradient methods. Experiments on the benchmark data set show that VarNDRR can achieve comparable results against state- of-the-art baselines without using any manual features. 1 Introduction Discourse relation characterizes the internal struc- ture and logical relation of a coherent text. Automat- ically identifying these relations not only plays an important role in discourse comprehension and gen- eration, but also obtains wide applications in many ∗Corresponding author other relevant natural language processing tasks, such as text summarization (Yoshida et al., 2014), conversation (Higashinaka et al., 2014), question an- swering (Verberne et al., 2007) and information ex- traction (Cimiano et al., 2005). Generally, discourse relations can be divided into two categories: explicit and implicit, which can be illustrated in the follow- ing example: The company was disappointed by the rul- ing. because The obligation is totally un- warranted. (adapted from wsj 0294) With the discourse connective because, these two sentences display an explicit discourse relation CONTINGENCY which can be inferred easily. Once this discourse connective is removed, however, the discourse relation becomes implicit and difficult to be recognized. This is because almost no surface in- formation in these two sentences can signal this re- lation. For successful recognition of this relation, in the contrary, we need to understand the deep seman- tic correlation between disappointed and obligation in the two sentences above. Although explicit dis- course relation recognition (DRR) has made great progress (Miltsakaki et al., 2005; Pitler et al., 2008), implicit DRR still remains a serious challenge due to the difficulty in semantic analysis. Conventional approaches to implicit DRR often treat the relation recognition as a classification prob- lem, where …

Content truncated to 500 words