NTNU: Measuring Semantic Similarity with Sublexical Feature Representations and Soft Cardinality

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 448–453, Dublin, Ireland, August 23-24, 2014. NTNU: Measuring Semantic Similarity with Sublexical Feature Representations and Soft Cardinality Andr´e Lynum, Partha Pakray, Bj¨orn Gamb¨ack Sergio Jimenez {andrely,parthap,gamback}@idi.ntnu.no sgjimenezv@unal.edu.co Norwegian University of Science and Technology Universidad Nacional de Colombia Trondheim, Norway Bogot´a, Colombia Abstract The paper describes the approaches taken by the NTNU team to the SemEval 2014 Semantic Textual Similarity shared task. The solutions combine measures based on lexical soft cardinality and character n-gram feature representations with lexi- cal distance metrics from TakeLab’s base- line system. The final NTNU system is based on bagged support vector machine regression over the datasets from previous shared tasks and shows highly competi- tive performance, being the best system on three of the datasets and third best overall (on weighted mean over all six datasets). 1 Introduction The Semantic Textual Similarity (STS) shared task aims at providing a unified framework for evaluat- ing textual semantic similarity, ranging from ex- act semantic equivalence to completely unrelated texts. This is represented by the prediction of a similarity score between two sentences, drawn from a particular category of text, which ranges from 0 (different topics) to 5 (exactly equivalent) through six grades of semantic similarity (Agirre et al., 2013). This paper describes the NTNU submission to the SemEval 2014 STS shared task (Task 10). The approach is based on the lexical and distributional features of the baseline Take- Lab system from the 2012 shared task (ˇSari´c et al., 2012), but improves on it in three ways: by adding two new categories of features and by using a bag- ging regression model to predict similarity scores. The new feature categories added are based on soft cardinality and character n-grams, described This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence de- tails:http://creativecommons.org/licenses/by/4.0/ in Section 2. The parameters of the two cate- gories are optimised over several corpora and the features are combined through support vector re- gression (Section 3) to create the actual systems (Section 4). As Section 5 shows, the new mea- sures give the baseline system a substantial boost, leading to very competitive results in the shared task evaluation. 2 Feature Generation Methods The methods used for creating new features utilise soft cardinality and character n-grams. Soft cardi- nality (Jimenez et al., 2010) was used successfully for the STS task in previous SemEval editions (Jimenez et al., 2012a; Jimenez et al., 2013a). The NTNU systems utilise an ensemble of such 18 measures, based only on surface text information, which were extracted using soft cardinality with different similarity functions, as further described in Section 2.1. Section 2.2 then introduces the similarity mea- sures based on character n-gram feature represen- tations, which proved themselves as the strongest features in the STS 2013 task (Marsi et al., 2013). The measures used here replace character n-gram features with cluster frequencies or vector val- ues …

Content truncated to 500 words