HHU at SemEval-2016 Task 1: Multiple Approaches to Measuring Semantic Textual Similarity

Proceedings of SemEval-2016, pages 595–601, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics HHU at SemEval-2016 Task 1: Multiple Approaches to Measuring Semantic Textual Similarity Matthias Liebeck, Philipp Pollack, Pashutan Modaresi, Stefan Conrad Institute of Computer Science Heinrich Heine University D¨usseldorf D-40225 D¨usseldorf, Germany {liebeck,modaresi,conrad}@cs.uni-duesseldorf.de philipp.pollack@hhu.de Abstract This paper describes our participation in the SemEval-2016 Task 1: Semantic Textual Sim- ilarity (STS). We developed three methods for the English subtask (STS Core). The first method is unsupervised and uses WordNet and word2vec to measure a token-based overlap. In our second approach, we train a neural net- work on two features. The third method uses word2vec and LDA with regression splines. 1 Introduction Measuring semantic textual similarity (STS) is the task of determining the similarity between two dif- ferent text passages. The task is important for var- ious natural language processing tasks like topic detection or automated text summarization because languages are versatile and authors can express sim- ilar content or even the same content with different words. Predicting semantic textual similarity has been a recurring task in SemEval challenges (Agirre et al., 2015; Agirre et al., 2014; Agirre et al., 2013; Agirre et al., 2012). As in previous years, the pur- pose of the STS task is the development of systems that automatically predict the semantic similarity of two sentences in the continuous interval [0, 5] where 0 represents a complete dissimilarity and 5 denotes a complete semantic equivalence between the sen- tences (Agirre et al., 2015). The organizers provide sentence pairs whose se- mantic similarities have to be predicted by the con- testants. The quality of a system is determined by calculating the Pearson correlation between the pre- dicted values and a human gold standard that has been created by crowdsourcing. The data from pre- vious STS tasks can be used for training supervised methods. The test data consists of text content from differ- ent sources. In this year’s shared task, the systems are tested on five different categories with different topics and varying textual characteristics like text length or spelling errors: answer-answer, plagia- rism, postediting, headlines, and question-question The remainder of the paper is structured as fol- lows: Section 2 discusses related approaches to au- tomatically determining semantic textual similarity. Section 3 describes our three methods in detail. We discuss their results in section 4. Finally, we con- clude in chapter 5 and outline future work. 2 Related Work In the last shared tasks, most of the teams used natural languages processing techniques like to- kenization, part-of-speech tagging, lemmatization, named entity recognition and word embeddings. Ex- ternal resources like WordNet (Miller, 1995) and word2vec (Mikolov et al., 2013) are commonly used. In (Agirre et al., 2012) and (Agirre et al., 2013), the organizers provide a list and a comparison of the tools and resources used by the participants in the first two years, respectively. In each year, the organizers provide a baseline value by calculating the cosine similarity of the bi- nary bag-of-words vectors from …

Content truncated to 500 words