DSS: Text Similarity Using Lexical Alignments of Form, Distributional Semantics and Grammatical Relations

DSS: Text Similarity Using Lexical Alignments of Form, Distributional Semantics and Grammatical Relations Diana McCarthy Saarland University diana@dianamccarthy.co.uk Spandana Gella University of Malta, Malta spandanagella@gmail.com Siva Reddy Lexical Computing Ltd, UK siva@sivareddy.in Abstract In this paper we present our systems for the STS task. Our systems are all based on a simple process of identifying the components that correspond between two sentences. Cur- rently we use words (that is word forms), lem- mas, distributional similar words and gram- matical relations identified with a dependency parser. We submitted three systems. All sys- tems only use open class words. Our first system (alignheuristic) tries to obtain a mapping between every open class token us- ing all the above sources of information. Our second system (wordsim) uses a different algorithm and unlike alignheuristic, it does not use the dependency information. The third system (average) simply takes the av- erage of the scores for each item from the other two systems to take advantage of the merits of both systems. For this reason we only provide a brief description of that. The results are promising, with Pearson’s coeffi- cients on each individual dataset ranging from .3765 to .7761 for our relatively simple heuris- tics based systems that do not require training on different datasets. We provide some anal- ysis of the results and also provide results for our data using Spearman’s, which as a non- parametric measure which we argue is better able to reflect the merits of the different sys- tems (average is ranked between the oth- ers). 1 Introduction Our motivation for the systems entered in the STS task (Agirre et al., 2012) was to model the contri- bution of each linguistic component of a sentence to the similarity of a candidate match and vice versa. Ultimately such a system could be exploited for ranking candidate paraphrases of a chunk of text of any length. We envisage a system as outlined in the future work section. The systems reported are simple baselines to such a system. We have two main systems (alignheuristic and wordsim) and also a system which simply uses the average score for each item from the two main systems (average). In our systems we: • only deal with open class words as to- kens i.e. nouns, verbs, adjectives, adverbs. alignheuristic and average also use numbers • assume that tokens have a 1:1 mapping • match: – word forms – lemmas – distributionally similar lemmas – alignheuristic and average use the grammatical relation with a word that has a mapping and the same relation in re- verse • score the sentence pair based on the size of the overlap. Different formulations of the score are used by our methods The paper is structured as follows. In the next section we make a brief mention of related work though of course there will be more pertinent related work presented and published at SemEval 2012. In section 3 we give a detailed account of the systems and in section 4 we provide …

Content truncated to 500 words