Textual Entailment with Structured Attentions and Composition

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2248–2258, Osaka, Japan, December 11-17 2016. Textual Entailment with Structured Attentions and Composition Kai Zhao and Liang Huang and Mingbo Ma School of Electrical Engineering and Computer Science Oregon State University Corvallis, Oregon, USA {kzhao.hf, lianghuang.sh, cosmmb}@gmail.com Abstract Deep learning techniques are increasingly popular in the textual entailment task, overcoming the fragility of traditional discrete models with hard alignments and logics. In particular, the recently proposed attention models (Rockt¨aschel et al., 2015; Wang and Jiang, 2015) achieves state-of-the-art accuracy by computing soft word alignments between the premise and hypothesis sentences. However, there remains a major limitation: this line of work completely ignores syntax and recursion, which is helpful in many traditional efforts. We show that it is beneficial to extend the attention model to tree nodes between premise and hypothesis. More importantly, this subtree-level attention reveals information about entailment relation. We study the recursive composition of this subtree-level entailment relation, which can be viewed as a soft version of the Natural Logic framework (MacCartney and Manning, 2009). Experiments show that our structured attention and entailment composition model can correctly identify and infer entailment relations from the bottom up, and bring significant improvements in accuracy. 1 Introduction Automatically recognizing sentence entailment relations between a pair of sentences has long been be- lieved to be an ideal testbed for discrete approaches using alignments and rigid logic inferences (Zanzotto et al., 2009; MacCartney and Manning, 2009; Wang and Manning, 2010; Watanabe et al., 2012; Tian et al., 2014; Filice et al., 2015). All of these methods are based on sparse features, making them brittle for unseen phrases and sentences. Recent advances in deep learning reveal another promising direction to solve this problem. Instead of discrete features and logics, continuous representation of the sentence is more robust to unseen features without sacrificing performance (Bowman et al., 2015). In particular, the attention model based on LSTM can successfully identify the word-by-word correspondences between the two sentences that lead to entailment or contradiction, which makes the entailment relation inference more focused on local information and less vulnerable to misleading information from other parts of the sentence (Rockt¨aschel et al., 2015; Wang and Jiang, 2015). However, conventional neural attention models for entailment recognition problem treat sentences as sequences, ignoring the fact that sentences are formed from the bottom up with syntactic tree structures, which inherently associate with the semantic meanings. Thus, using the tree structure of the sentences will be beneficial in inducing the entailment relations between parts of the two sentences, and then further improving the sentence-level entailment relation classification (Watanabe et al., 2012). Furthermore, as MacCartney and Manning (2009) point out, the entailment relation between sentences is modular, and can be modeled as the composition of subtree-level entailment relations. These subtree- level entailment relations are induced by comparing subtrees between the two sentences, which are by nature a perfect match to be modeled by the attention model over trees. In this …

Content truncated to 500 words