UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures

First Joint Conference on Lexical and Computational Semantics (*SEM), pages 435–440, Montr´eal, Canada, June 7-8, 2012. c⃝2012 Association for Computational Linguistics UKP: Computing Semantic Textual Similarity by Combining Multiple Content Similarity Measures Daniel B¨ar†, Chris Biemann†, Iryna Gurevych†‡, and Torsten Zesch†‡ †Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universit¨at Darmstadt ‡Ubiquitous Knowledge Processing Lab (UKP-DIPF) German Institute for Educational Research and Educational Information www.ukp.tu-darmstadt.de Abstract We present the UKP system which performed best in the Semantic Textual Similarity (STS) task at SemEval-2012 in two out of three met- rics. It uses a simple log-linear regression model, trained on the training data, to combine multiple text similarity measures of varying complexity. These range from simple char- acter and word n-grams and common sub- sequences to complex features such as Ex- plicit Semantic Analysis vector comparisons and aggregation of word similarity based on lexical-semantic resources. Further, we em- ploy a lexical substitution system and statisti- cal machine translation to add additional lex- emes, which alleviates lexical gaps. Our final models, one per dataset, consist of a log-linear combination of about 20 features, out of the possible 300+ features implemented. 1 Introduction The goal of the pilot Semantic Textual Similarity (STS) task at SemEval-2012 is to measure the de- gree of semantic equivalence between pairs of sen- tences. STS is fundamental to a variety of tasks and applications such as question answering (Lin and Pantel, 2001), text reuse detection (Clough et al., 2002) or automatic essay grading (Attali and Burstein, 2006). STS is also closely related to tex- tual entailment (TE) (Dagan et al., 2006) and para- phrase recognition (Dolan et al., 2004). It differs from both tasks, though, insofar as those operate on binary similarity decisions while STS is defined as a graded notion of similarity. STS further requires a bidirectional similarity relationship to hold between a pair of sentences rather than a unidirectional en- tailment relation as for the TE task. A multitude of measures for computing similar- ity between texts have been proposed in the past based on surface-level and/or semantic content fea- tures (Mihalcea et al., 2006; Landauer et al., 1998; Gabrilovich and Markovitch, 2007). The exist- ing measures exhibit two major limitations, though: Firstly, measures are typically used in separation. Thereby, the assumption is made that a single measure inherently captures all text characteristics which are necessary for computing similarity. Sec- ondly, existing measures typically exclude similar- ity features beyond content per se, thereby implying that similarity can be computed by comparing text content exclusively, leaving out any other text char- acteristics. While we can only briefly tackle the sec- ond issue here, we explicitly address the first one by combining several measures using a supervised ma- chine learning approach. With this, we hope to take advantage of the different facets and intuitions that are captured in the single measures. In the following section, we describe the feature space in detail. Section 3 describes the machine learning setup. After describing our submitted runs, we discuss …

Content truncated to 500 words