ExB Themis: Extensive Feature Extraction from Word Alignments for Semantic Textual Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 264–268, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics ExB Themis: Extensive Feature Extraction from Word Alignments for Semantic Textual Similarity Christian H¨anig, Robert Remus, Xose De La Puente ExB Research & Development GmbH Seeburgstr. 100 04103 Leipzig, Germany {haenig, remus, puente}@exb.de Abstract We present ExB Themis – a word alignment- based semantic textual similarity system de- veloped for SemEval-2015 Task 2: Semantic Textual Similarity. It combines both string and semantic similarity measures as well as align- ment features using Support Vector Regres- sion. It occupies the first three places on Span- ish data and additionally places second on En- glish data. ExB Themis proved to be the best multilingual system among all participants. 1 Introduction Semantic Textual Similarity (STS) is the task of measuring the degree of semantic equivalence of a sentence pair and is applicable to problems in Ma- chine Translation and Summarization among others (Agirre et al., 2012). STS has drawn a lot of atten- tion in the last few years leading to the availabil- ity of multilingual training and test data and to the development of a variety of approaches. These ap- proaches fall broadly into three categories (Han et al., 2013): Vector space approaches: Texts are represented as bag-of-words vectors and a vector similarity – e. g. cosine – is used to compute a similarity score between two texts (Meadow et al., 1992). Alignment approaches: Words and phrases in two texts are aligned and the quality or coverage of the resulting alignments are used as similarity measure (Mihalcea et al., 2006; Sultan et al., 2014). Machine Learning approaches: Multiple similar- ity measures and features are combined using supervised Machine Learning (ML). This ap- proach relies on the availability of training data (B¨ar et al., 2012; ˇSari´c et al., 2012). ExB Themis combines advantages of all three cat- egories: we implemented a complex alignment al- gorithm focusing on named entities, temporal ex- pressions, measurement expressions and dedicated negation handling. Unlike other alignment-based approaches, we extract a variety of features to better model the properties of alignments instead of pro- viding only one alignment feature (see Section 4.1). Moreover, we employ a variety of similarity mea- sures based on strings and lexical items (see Sec- tion 4.2). Our system integrates two well-known language resources – WordNet1 and ConceptNet (Speer and Havasi, 2012). Additionally, it uses word embeddings to cope with data sparseness and the in- sufficiency of overlaps between sentences. Finally, we train a Support Vector Regression (SVR) model using these features (see Section 5). 2 Preprocessing Our text preprocessing comprises tokenization, case correction (e. g. US Flying Surveillance Missions to Help Find Kidnapped Nigerian Girls is corrected to US flying surveillance missions to help find kid- napped Nigerian girls), unsupervised part-of-speech (POS) tagging based on SVD2 (Lamar et al., 2010), 1English: we use the one described by Miller (1995); Span- ish: we use the one presented in (Gonz´alez-Agirre et al., 2012). 264 …

Content truncated to 500 words