Modelling Interaction of Sentence Pair with Coupled-LSTMs

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1703–1712, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Modelling Interaction of Sentence Pair with Coupled-LSTMs Pengfei Liu Xipeng Qiu∗ Yaqian Zhou Jifan Chen Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {pfliu14,xpqiu,zhouyaqian,jfchen14, xjhuang}@fudan.edu.cn Abstract Recently, there is rising interest in modelling the interactions of two sentences with deep neural networks. However, most of the exist- ing methods encode two sequences with sepa- rate encoders, in which a sentence is encoded with little or no information from the other sentence. In this paper, we propose a deep architecture to model the strong interaction of sentence pair with two coupled-LSTMs. Specifically, we introduce two coupled ways to model the interdependences of two LSTMs, coupling the local contextualized interactions of two sentences. We then aggregate these in- teractions and use a dynamic pooling to select the most informative features. Experiments on two very large datasets demonstrate the effi- cacy of our proposed architectures. 1 Introduction Distributed representations of words or sentences have been widely used in many natural language processing (NLP) tasks, such as text classification (Kalchbrenner et al., 2014; Liu et al., 2015), ques- tion answering and machine translation (Sutskever et al., 2014) and so on. Among these tasks, a com- mon problem is modelling the relevance/similarity of the sentence pair, which is also called text seman- tic matching. Recently, deep learning based models is rising a substantial interest in text semantic matching and have achieved some great progresses (Hu et al., 2014; Qiu and Huang, 2015; Wan et al., 2016). ∗Corresponding author. According to the phases of interaction between two sentences, previous models can be classified into three categories. Weak interaction Models Some early works fo- cus on sentence level interactions, such as ARC- I(Hu et al., 2014), CNTN(Qiu and Huang, 2015) and so on. These models first encode two sequences with some basic (Neural Bag-of-words, BOW) or advanced (RNN, CNN) components of neural net- works separately, and then compute the matching score based on the distributed vectors of two sen- tences. In this paradigm, two sentences have no in- teraction until arriving final phase. Semi-interaction Models Some improved meth- ods focus on utilizing multi-granularity represen- tation (word, phrase and sentence level), such as MultiGranCNN (Yin and Sch¨utze, 2015) and Multi- Perspective CNN (He et al., 2015). Another kind of models use soft attention mechanism to obtain the representation of one sentence by depending on representation of another sentence, such as ABCNN (Yin et al., 2015), Attention LSTM(Rockt¨aschel et al., 2015; Hermann et al., 2015). These models can alleviate the weak interaction problem, but are still insufficient to model the contextualized interaction on the word as well as phrase level. Strong Interaction Models These models di- rectly build an interaction space between two sen- tences and model the interaction at different posi- tions, such as ARC-II (Hu et al., 2014), MV-LSTM …

Content truncated to 500 words