BUAP: Evaluating Features for Multilingual and Cross-Level Semantic Textual Similarity

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 149–153, Dublin, Ireland, August 23-24, 2014. BUAP: Evaluating Features for Multilingual and Cross-Level Semantic Textual Similarity Darnes Vilari˜no, David Pinto, Sa´ul Le´on, Mireya Tovar,Beatriz Beltr´an Benem´erita Universidad Aut´onoma de Puebla Faculty of Computer Science 14 Sur y Av. San Claudio, CU Puebla, Puebla, M´exico {darnes,dpinto,saul.leon,mtovar,bbeltran}@cs.buap.mx Abstract In this paper we present the evaluation of different features for multiligual and cross- level semantic textual similarity. Three dif- ferent types of features were used: lexical, knowledge-based and corpus-based. The re- sults obtained at the Semeval competition rank our approaches above the average of the rest of the teams highlighting the usefulness of the features presented in this paper. 1 Introduction Semantic textual similarity aims to capture whether the meaning of two texts are similar. This concept is somehow different from the textual similarity def- inition itself, because in the latter we are only in- terested in measuring the number of lexical com- ponents that the two texts share. Therefore, tex- tual similarity can range from exact semantic equiv- alence to a complete unrelatedness pair of texts. Finding the semantic similarity between a pair of texts has become a big challenge for specialists in Natural Language Processing (NLP), because it has applications in some NLP task such as machine translation, automatic construction of summaries, authorship attribution, machine reading comprehen- sion, information retrieval, among others, which usually need a manner to calculate degrees of simi- larity between two given texts. This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ Semantic textual similarity can be calculated us- ing texts of different sizes, for example between, a paragraph and a sentence, or a sentence and a phrase, or a phrase and a word, or even a word and a sense. When we consider this difference, we say the task is called “Cross-Level Semantic Similarity”, but when this distinction is not considered, then we call the task just as “Semantic Textual Similarity”. In this paper, we evaluate different features for de- termining those that obtain the best performances for calculating both, cross-level semantic similarity and multilingual semantic textual similarity. The remaining of this paper is structured as fol- lows. Section 2 presents the features used in both experiments. Section 3 shows the manner we used the features for determining the degree of seman- tic textual similarity. Section 4, on the other hand, shows the experiments we have carried out for de- termining cross-level semantic similarity. Finally, in Section 5 the conclusions and findings are given. 2 Description of Features In this section we describe the different features used for evaluation semantic textual similarity. Basically, we have used three different types of features: lex- ical, knowledge-based and corpus-based. The first one, counts the frequency of ocurrence of lexical features which include n-grams of characters, skip- grams1, words and some lexical relationships such as synonymy or hypernymy. Additionally, we …

Content truncated to 500 words