FBK-TR: Applying SVM with Multiple Linguistic Features for Cross-Level Semantic Similarity

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 284–288, Dublin, Ireland, August 23-24, 2014. FBK-TR: Applying SVM with Multiple Linguistic Features for Cross-Level Semantic Similarity Ngoc Phuoc An Vo Fondazione Bruno Kessler University of Trento Trento, Italy ngoc@fbk.eu Tommaso Caselli TrentoRISE Trento, Italy t.caselli@trentorise.eu Octavian Popescu Fondazione Bruno Kessler Trento, Italy popescu@fbk.eu Abstract Recently, the task of measuring seman- tic similarity between given texts has drawn much attention from the Natural Language Processing community. Espe- cially, the task becomes more interesting when it comes to measuring the seman- tic similarity between different-sized texts, e.g paragraph-sentence, sentence-phrase, phrase-word, etc. In this paper, we, the FBK-TR team, describe our system par- ticipating in Task 3 "Cross-Level Seman- tic Similarity", at SemEval 2014. We also report the results obtained by our system, compared to the baseline and other partic- ipating systems in this task. 1 Introduction Measuring semantic text similarity has become a hot trend in NLP as it can be applied to other tasks, e.g. Information Retrieval, Paraphrasing, Machine Translation Evaluation, Text Summariza- tion, Question and Answering, and others. Several approaches proposed to measure the semantic sim- ilarity between given texts. The first approach is based on vector space models (VSMs) (Meadow, 1992). A VSM transforms given texts into "bag- of-words" and presents them as vectors. Then, it deploys different distance metrics to compute the closeness between vectors, which will return as the distance or similarity between given texts. The next well-known approach is using text-alignment. By assuming that two given texts are semantically similar, they could be aligned on word or phrase levels. The alignment quality can serve as a simi- larity measure. "It typically pairs words from the two texts by maximizing the summation of the This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ word similarity of the resulting pairs" (Mihalcea et al., 2006). In contrast, the third approach uses machine learning techniques to learn models built from different lexical, semantic and syntactic fea- tures and then give predictions on degree of simi- larity between given texts (Šari´c et al., 2012). At SemEval 2014, the Task 3 "Cross-Level Se- mantic Similarity" (Jurgens et al., 2014) is to eval- uate the semantic similarity across different sizes of texts, in particular, a larger-sized text is com- pared to a smaller-sized one. The task consists of four types of semantic similarity comparison: paragraph to sentence, sentence to phrase, phrase to word, and word to sense. The degree of similar- ity ranges from 0 (different meanings) to 4 (simi- lar meanings). For evaluation, systems were eval- uated, first, within comparison type and second, across all comparison types. Two methods are used to evaluate between system outputs and gold standard (human annotation), which are Pearson correlation and Spearman’s rank correlation (rho). The FBK-TR team participated in this task with three different runs. In this paper, we present a clear and comprehensive description …

Content truncated to 500 words