Deep Fusion LSTMs for Text Semantic Matching

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1034–1043, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Deep Fusion LSTMs for Text Semantic Matching Pengfei Liu, Xipeng Qiu∗, Jifan Chen, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {pfliu14,xpqiu,jfchen14,xjhuang}@fudan.edu.cn Abstract Recently, there is rising interest in mod- elling the interactions of text pair with deep neural networks. In this paper, we propose a model of deep fusion LSTMs (DF-LSTMs) to model the strong inter- action of text pair in a recursive match- ing way. Specifically, DF-LSTMs con- sist of two interdependent LSTMs, each of which models a sequence under the in- fluence of another. We also use exter- nal memory to increase the capacity of LSTMs, thereby possibly capturing more complicated matching patterns. Experi- ments on two very large datasets demon- strate the efficacy of our proposed archi- tecture. Furthermore, we present an elab- orate qualitative analysis of our models, giving an intuitive understanding how our model worked. 1 Introduction Among many natural language processing (NLP) tasks, such as text classification, question answer- ing and machine translation, a common problem is modelling the relevance/similarity of a pair of texts, which is also called text semantic matching. Due to the semantic gap problem, text semantic matching is still a challenging problem. Recently, deep learning is rising a substan- tial interest in text semantic matching and has achieved some great progresses (Hu et al., 2014; Qiu and Huang, 2015; Wan et al., 2016). Accord- ing to their interaction ways, previous models can be classified into three categories: Weak interaction Models Some early works focus on sentence level interactions, such as ARC- I(Hu et al., 2014), CNTN(Qiu and Huang, 2015) ∗Corresponding author and so on. These models first encode two se- quences into continuous dense vectors by sepa- rated neural models, and then compute the match- ing score based on sentence encoding. In this paradigm, two sentences have no interaction un- til arriving final phase. Semi-interaction Models Another kind of mod- els use soft attention mechanism to obtain the rep- resentation of one sentence by depending on rep- resentation of another sentence, such as ABCNN (Yin et al., 2015), Attention LSTM (Rockt¨aschel et al., 2015; Hermann et al., 2015). These models can alleviate the weak interaction problem to some extent. Strong Interaction Models Some models build the interaction at different granularity (word, phrase and sentence level), such as ARC-II (Hu et al., 2014), MultiGranCNN (Yin and Sch¨utze, 2015), Multi-Perspective CNN (He et al., 2015), MV-LSTM (Wan et al., 2016), MatchPyramid (Pang et al., 2016). The final matching score de- pends on these different levels of interactions. In this paper, we adopt a deep fusion strat- egy to model the strong interactions of two sen- tences. Given two texts x1:m and y1:n, we define a matching vector hi,j to represent the interaction of the subsequences x1:i and y1:j. hi,j depends on the matching …

Content truncated to 500 words