Inter-Weighted Alignment Network for Sentence Pair Modeling

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1179–1189 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Inter-Weighted Alignment Network for Sentence Pair Modeling Gehui Shen Yunlun Yang Zhi-Hong Deng∗ Key Laboratory of Machine Perception (Ministry of Education), School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China {jueliangguke, incomparable-lun, zhdeng}@pku.edu.cn Abstract Sentence pair modeling is a crucial prob- lem in the field of natural language pro- cessing. In this paper, we propose a model to measure the similarity of a sentence pair focusing on the interaction informa- tion. We utilize the word level similarity matrix to discover fine-grained alignment of two sentences. It should be emphasized that each word in a sentence has a different importance from the perspective of seman- tic composition, so we exploit two novel and efficient strategies to explicitly calcu- late a weight for each word. Although the proposed model only use a sequen- tial LSTM for sentence modeling with- out any external resource such as syntactic parser tree and additional lexicon features, experimental results show that our model achieves state-of-the-art performance on three datasets of two tasks. 1 Introduction Given two pieces of sentences S and T, sentence pair modeling (SPM) is a fundamental task whose applications include question answering (Lin, 2007), natural language inference (Bowman et al., 2015), paraphrase identification (Socher et al., 2011a) and sentence completion (Wan et al., 2016) and so on. In general, each of the two sentences are firstly mapped to a representation, and then a model is designed to determine the relation be- tween them. Traditional methods use lexicon fea- tures such as Bag-of-Words(BOW) to map sen- tences. As we know, features design and selection are time-consuming and high dimensional features may suffer from sparsity because of the varia- tion of linguistic. Recently, deep learning tech- ∗Corresponding author niques have been applied to develop end-to-end models for NLP tasks, such as sentence model- ing (Socher et al., 2011b; Kim, 2014), relation classification (Socher et al., 2012) and machine translation (Sutskever et al., 2014). These works show that deep learning models can be compara- ble with hand-crafted features based models and often outperform them. Existing DNN models are based on pre-trained word embeddings which map each word to one low dimensional vector and compose word em- beddings to represent sentence. Some models are developed directly from the sentence models. They obtain single vector representation for each sentence separately and then determine the rela- tion based on two vectors (Huang et al., 2013; Qiu and Huang, 2015; Palangi et al., 2016). Because of the absence of interaction, these models can not achieve state-of-the-art performance. Inspired by attention mechanism in computer vision and machine translation, some elaborate models have been proposed (Rockt¨aschel et al., 2016; Zhou et al., 2016; Wang and Jiang, 2016) which take interaction information into consid- eration. Meanwhile, to grasp the fine-grained information for semantic similarity, some prior works (Pang et al., 2016; He and Lin, 2016) firstly compute …

Content truncated to 500 words