TATO: Leveraging on Multiple Strategies for Semantic Textual Similarity

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 190–195, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics TATO: Leveraging on Multiple Strategies for Semantic Textual Similarity Tu Thanh Vu†, Quan Hung Tran††, Son Bao Pham† †University of Engineering and Technology, Vietnam National University, Hanoi, Vietnam ††Japan Advanced Institute of Science and Technology, Japan † {tuvt,sonpb}@vnu.edu.vn †† quanth@jaist.ac.jp Abstract In this paper, we describe the TATO sys- tem which participated in the SemEval-2015 Task 2a: “Semantic Textual Similarity (STS) for English”. Our system is trained on pub- lished datasets from the previous competi- tions. Based on some machine learning tech- niques, it combines multiple similarity mea- sures of varying complexity ranging from sim- ple lexical and syntactic similarity measures to complex semantic similarity ones to com- pute semantic textual similarity. Our final model consists of a simple linear combination of about 30 main features out of a numerous number of features experimented. The results are promising, with Pearson’s coefficients on each individual dataset ranging from 0.6796 to 0.8167 and an overall weighted mean score of 0.7422, well above the task baseline system. 1 Introduction Measuring semantic textual similarity (STS) can be defined as the task of computing the degree of se- mantic equivalence between pairs of texts. It has drawn an increasing amount of attention from the NLP community, especially at level of short text fragments, as partly reflected in the SemEval tasks in recent years. In the SemEval-2015 Task 2, the de- gree of semantic equivalence for each sentence pair is represented by a similarity score between 0 (no relation) and 5 (semantic equivalence). STS has a wide range of applications which includes applica- tions for machine translation evaluation, information extraction, question answering, and summarization. STS is related to, but different from textual en- tailment (TE) (Dagan et al., 2006) and paraphrase recognition (PARA) (Dolan et al., 2004) as it aims to render a graded notion of semantic equivalence between two textual snippets, rather than a binary yes/no decision. STS requires a bidirectional sim- ilarity relation between sentences, while TE anno- tates them with an unidirectional entailment relation. The literature of STS is rife with attempts to compute similarity between texts using a multitude of measures at different levels of depth: lexical (Malakasiotis and Androutsopoulos, 2007), syntac- tic (Malakasiotis, 2009; Zanzotto et al., 2009), and semantic (Rinaldi et al., 2003; Bos and Markert, 2005). (Gomaa and Fahmy, 2013) discusses exist- ing works on STS and partitions them into three cat- egories based on the similarity measures used: (i) string-based approaches (B¨ar et al., 2012; Malakasi- otis and Androutsopoulos, 2007) which operate on string sequences and character composition to com- pute similarities and can be categorized into two groups: character-based and term-based approaches; (ii) corpus-based approaches (Li et al., 2006) which gain statistics information about words from large corpora and reflect their semantics in distributional high semantic space to determine the similarity, such as Latent Semantic Analysis (LSA) (Landauer et al., 1998; Foltz et al., …

Content truncated to 500 words