HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 130–133, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics HCTI at SemEval-2017 Task 1: Use convolutional neural network to evaluate Semantic Textual Similarity Yang SHAO Hitachi, Ltd. / Higashi koigakubo 1-280, Kokubunji-shi, Tokyo, Japan yang.shao.kn@hitachi.com Abstract This paper describes our convolutional neural network (CNN) system for the Se- mantic Textual Similarity (STS) task. We calculated semantic similarity score be- tween two sentences by comparing their semantic vectors. We generated a seman- tic vector by max pooling over every di- mension of all word vectors in a sentence. There are two key design tricks used by our system. One is that we trained a CNN to transfer GloVe word vectors to a more proper form for the STS task be- fore pooling. Another is that we trained a fully-connected neural network (FCNN) to transfer the difference of two seman- tic vectors to the probability distribution over similarity scores. All hyperparame- ters were empirically tuned. In spite of the simplicity of our neural network system, we achieved a good accuracy and ranked 3rd on primary track of SemEval 2017. 1 Introduction Semantic Textual Similarity (STS) is the task of determining the degree of semantic similarity between two sentences. STS task is a building block of many natural language processing (NLP) applications. Therefore, it has received a signif- icant amount of attention in recent years. STS tasks in SemEval have been held from 2012 to 2017 (Cer et al., 2017). Successfully estimat- ing the degree of semantic similarity between two sentences requires a very deep understanding of both sentences. Well performing STS methods can be applied to many other natural language un- derstanding tasks including paraphrasing, entail- ment detection, answer selection, hypothesis evi- dencing, machine translation (MT) evaluation and quality estimation, summarization, question an- swering (QA) and short answer grading. Measuring sentence similarity is challenging for two reasons. One is the variability of linguistic expression and the other is the limited amount of annotated training data. Therefore, conventional NLP approaches, such as sparse, hand-crafted fea- tures are difficult to use. However, neural network systems (He et al., 2015a; He and Lin, 2016) can alleviate data sparseness with pre-training and dis- tributed representations. We propose a convolu- tional neural network system with 5 components: 1) Enhance GloVe word vectors by adding hand- crafted features. 2) Transfer the enhanced word vectors to a more proper form by a convolutional neural network. 3) Max pooling over every dimension of all word vectors to generate semantic vector. 4) Generate semantic difference vector by con- catenating the element-wise absolute differ- ence and the element-wise multiplication of two semantic vectors. 5) Transfer the semantic difference vector to the probability distribution over similarity scores by fully-connected neural network. 2 System Description Figure 1 provides an overview of our system. The two sentences to be semantically compared are first pre-processed as described in subsection 2.1. Then the CNN described in subsection 2.2 combines the word vectors …

Content truncated to 500 words