A Sentence Interaction Network for Modeling Dependence between Sentences

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 558–567, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A Sentence Interaction Network for Modeling Dependence between Sentences Biao Liu1, Minlie Huang1∗, Song Liu2, Xuan Zhu2, Xiaoyan Zhu1 1State Key Lab. of Intelligent Technology and Systems 1National Lab. for Information Science and Technology 1Dept. of Computer Science and Technology, Tsinghua University, Beijing, China 2Samsung R&D Institute, Beijing, China 1liubiao2638@gmail.com, {aihuang, zxy-dcs}@tsinghua.edu.cn Abstract Modeling interactions between two sen- tences is crucial for a number of natu- ral language processing tasks including Answer Selection, Dialogue Act Analy- sis, etc. While deep learning methods like Recurrent Neural Network or Convo- lutional Neural Network have been proved to be powerful for sentence modeling, prior studies paid less attention on inter- actions between sentences. In this work, we propose a Sentence Interaction Net- work (SIN) for modeling the complex in- teractions between two sentences. By in- troducing “interaction states” for word and phrase pairs, SIN is powerful and flexi- ble in capturing sentence interactions for different tasks. We obtain significant im- provements on Answer Selection and Dia- logue Act Analysis without any feature en- gineering. 1 Introduction There exist complex interactions between sen- tences in many natural language processing (NLP) tasks such as Answer Selection (Yu et al., 2014; Yin et al., 2015), Dialogue Act Analysis (Kalch- brenner and Blunsom, 2013), etc. For instance, given a question and two candidate answers below, though they are all talking about cats, only the first Q What do cats look like? A1 Cats have large eyes and furry bodies. A2 Cats like to play with boxes and bags. answer correctly answers the question about cats’ appearance. It is important to appropriately model the relation between two sentences in such cases. ∗ Correspondence author For sentence pair modeling, some methods first project the two sentences to fix-sized vectors sep- arately without considering the interactions be- tween them, and then fed the sentence vectors to other classifiers as features for a specific task (Kalchbrenner and Blunsom, 2013; Tai et al., 2015). Such methods suffer from being unable to encode context information during sentence em- bedding. A more reasonable way to capture sentence in- teractions is to introduce some mechanisms to uti- lize information from both sentences at the same time. Some methods attempt to introduce an at- tention matrix which contains similarity scores be- tween words and phrases to approach sentence in- teractions (Socher et al., 2011; Yin et al., 2015). While the meaning of words and phrases may drift from contexts to contexts, simple similarity scores may be too weak to capture the complex interac- tions, and a more powerful interaction mechanism is needed. In this work, we propose a Sentence Interaction Network (SIN) focusing on modeling sentence in- teractions. The main idea behind this model is that each word in one sentence may potentially in- fluence every word in another sentence in some degree (the word “influence” here may refer to “answer” or …

Content truncated to 500 words