Element-wise Bilinear Interaction for Sentence Matching

Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (*SEM), pages 107–112 New Orleans, June 5-6, 2018. c⃝2018 Association for Computational Linguistics Element-wise Bilinear Interaction for Sentence Matching Jihun Choi, Taeuk Kim, Sang-goo Lee Department of Computer Science and Engineering Seoul National University, Seoul, Korea {jhchoi,taeuk,sglee}@europa.snu.ac.kr Abstract When we build a neural network model pre- dicting the relationship between two sen- tences, the most general and intuitive approach is to use a Siamese architecture, where the sentence vectors obtained from a shared en- coder is given as input to a classifier. For the classifier to work effectively, it is important to extract appropriate features from the two vectors and feed them as input. There exist several previous works that suggest heuristic- based function for matching sentence vectors, however it cannot be said that the heuris- tics tailored for a specific task generalize to other tasks. In this work, we propose a new matching function, ElBiS, that learns to model element-wise interaction between two vectors. From experiments, we empirically demonstrate that the proposed ElBiS matching function outperforms the concatenation-based or heuristic-based matching functions on nat- ural language inference and paraphrase identi- fication, while maintaining the fused represen- tation compact. 1 Introduction Identifying the relationship between two sentences is a key component for various natural language processing tasks such as paraphrase identification, semantic relatedness prediction, textual entailment recognition, etc. The most general and intuitive approach to these problems would be to encode each sentence using a sentence encoder network and feed the encoded vectors to a classifier net- work.1 For a model to predict the relationship correctly, it is important for the input to the classifier to contain appropriate information. The most na¨ıve 1 The encoded vectors can also be fed into a regression network, however in this work we focus only on classifica- tion. method is to concatenate the two vectors and del- egate the role of extracting features to subsequent network components. However, despite the theo- retical fact that even a single-hidden layer feedfor- ward network can approximate any arbitrary func- tion (Cybenko, 1989; Hornik, 1991), the space of network parameters is too large, and it is helpful to narrow down the search space by directly giv- ing information about interaction to the classifier model, as empirically proven in previous works built for various tasks (Ji and Eisenstein, 2013; Mou et al., 2016; Xiong et al., 2016, to name but a few). In this paper, we propose a matching function which learns from data to fuse two sentence vec- tors and extract useful features. Unlike bilinear pooling methods designed for matching vectors from heterogeneous domain (e.g. image and text), our proposed method utilizes element-wise bilin- ear interaction between vectors rather than interdi- mensional interaction. In §3, we will describe the intuition and assumption behind the restriction of interaction. This paper is organized as follows. In §2, we briefly introduce previous work related to our ob- jective. The detailed explanation of the proposed model is given in §3, and …

Content truncated to 500 words