BUAP: Evaluating Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 145–148, Dublin, Ireland, August 23-24, 2014. BUAP: Evaluating Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment Sa´ul Le´on, Darnes Vilari˜no, David Pinto, Mireya Tovar, Beatriz Beltr´an Benem´erita Universidad Aut´onoma de Puebla Faculty of Computer Science 14 Sur y Av. San Claudio, CU Puebla, Puebla, M´exico {saul.leon,darnes,dpinto,mtovar,bbeltran}@cs.buap.mx Abstract The results obtained by the BUAP team at Task 1 of SemEval 2014 are presented in this paper. The run submitted is a supervised ver- sion based on two classification models: 1) We used logistic regression for determining the semantic relatedness between a pair of sentences, and 2) We employed support vec- tor machines for identifying textual entailment degree between the two sentences. The be- haviour for the second subtask (textual entail- ment) obtained much better performance than the one evaluated at the first subtask (related- ness), ranking our approach in the 7th position of 18 teams that participated at the competi- tion. 1 Introduction The Compositional Distributional Semantic Models (CDSM) applied to sentences aim to approximate the meaning of those sentences with vectors summa- rizing their patterns of co-occurrence in corpora. In the Task 1 of SemEval 2014, the organizers aimed to evaluate the performance of this kind of models through the following two tasks: semantic related- ness and textual entailment. Semantic relatedness captures the degree of semantic similarity, in this case, between a pair of sentences, whereas textual entailment allows to determine the entailment rela- tion holding between two sentences. This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ This document is a description paper, therefore, we focus the rest of it on the features and models we used for carrying out the experiments. A complete description of the task and the dataset used are given in Marelli et al. (2014a) and in Marelli et al. (2014b), respectively. The remaining of this paper is structured as fol- lows. In Section 2 we describe the general model we used for comparing two sentences and the set of the features used for constructing the vectorial rep- resentation for each sentence. Section 3 shows how we integrate the features calculated in a single vector which fed a supervised classifier aiming to construct a classication model that solves the two aforemen- tioned problems: semantic relatedness and textual entailment. In the same section we show the ob- tained results. Finally, in Section 4 we present our findings. 2 Description of the Distributional Semantic Model Used Given a sentence S = w1w2 · · · w|S|, with wi a sen- tence word, we have calculated different correlated terms (ti,j) or a numeric vector (Vi) for each word wi as follows: 1. {ti,j|relation(ti,j, wi)} such as “relation” is one the following dependency relations: “ob- ject”, “subject” or “property”. 2. {ti,j|ti,j = ck · · · ck+n} with n = 2, · · · …

Content truncated to 500 words