UWB at SemEval-2016 Task 1: Semantic Textual Similarity using Lexical, Syntactic, and Semantic Information

Proceedings of SemEval-2016, pages 588–594, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics UWB at SemEval-2016 Task 1: Semantic Textual Similarity using Lexical, Syntactic, and Semantic Information Tom´aˇs Brychc´ın NTIS – New Technologies for the Information Society, Faculty of Applied Sciences, University of West Bohemia, Technick´a 8, 306 14 Plzeˇn Czech Republic brychcin@kiv.zcu.cz Luk´aˇs Svoboda Department of Computer Science and Engineering, Faculty of Applied Sciences, University of West Bohemia, Technick´a 8, 306 14 Plzeˇn Czech Republic svobikl@kiv.zcu.cz Abstract We present our UWB system for Seman- tic Textual Similarity (STS) task at SemEval 2016. Given two sentences, the system esti- mates the degree of their semantic similarity. We use state-of-the-art algorithms for the meaning representation and combine them with the best performing approaches to STS from previous years. These methods benefit from various sources of information, such as lexical, syntactic, and semantic. In the monolingual task, our system achieve mean Pearson correlation 75.7% compared with human annotators. In the cross-lingual task, our system has correlation 86.3% and is ranked first among 26 systems. 1 Introduction Semantic Textual Similarity (STS) is one of the core disciplines in Natural Language Processing (NLP). Assume we have two textual fragments (word phrases, sentences, paragraphs, or full doc- uments), the goal is to estimate the degree of their semantic similarity. STS systems are usually compared with the man- ually annotated data. In the case of SemEval the data consist of pairs of sentences with a score between 0 and 5 (higher number means higher semantic simi- larity). For example, English pair Two dogs play in the grass. Two dogs playing in the snow. has a score 2.8, i.e. the sentences are not equivalent, but share some information. This year, SemEval’s STS is extended with the Spanish-English cross-lingual subtask, where e.g. the pair Tuve el mismo problema que t´u. I had the same problem. has a score 4.8, which means nearly equivalent. Each year STS belongs to one of the most pop- ular tasks at SemEval competition. The best STS system at SemEval 2012 (B¨ar et al., 2012) used lexi- cal similarity and Explicit Semantic Analysis (ESA) (Gabrilovich and Markovitch, 2007). In SemEval 2013, the best model (Han et al., 2013) used se- mantic models such as Latent Semantic Analysis (LSA) (Deerwester et al., 1990), external informa- tion sources (WordNet) and n-gram matching tech- niques. For SemEval 2014 and 2015 the best sys- tem comes from (Sultan et al., 2014a; Sultan et al., 2014b; Sultan et al., 2015). They introduced new algorithm, which align the words between two sen- tences. They showed that this approach can be effi- ciently used also for STS. Overview of systems par- ticipating in previous SemEval competitions can be found in (Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014; Agirre et al., 2015). The best performing systems from previous years are based on various architectures benefiting from lexical, syntactic, and semantic information. In this work we try to use the best techniques presented dur- ing …

Content truncated to 500 words