UQeResearch: Semantic Textual Similarity Quantification

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 123–127, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics UQeResearch: Semantic Textual Similarity Quantification Hamed Hassanzadeh1, Tudor Groza1,2, Anthony Nguyen3, Jane Hunter1 1School of ITEE, The University of Queensland, St Lucia, QLD, Australia 2Garvan Institute of Medical Research, Darlinghurst, NSW, Australia 3Australian e-Health Research Centre, CSIRO, Brisbane, QLD, Australia h.hassanzadeh@uq.edu.au, t.groza@garvan.org.au, anthony.nguyen@csiro.au, jane@itee.uq.edu.au Abstract This paper presents an approach for estimat- ing the Semantic Textual Similarity of full English sentences as specified in Shared Task 2 of SemEval-2015. The semantic similarity of sentence pairs is quantified from three per- spectives - structural, syntactical, and seman- tic. The numerical representations of the derived similarity measures are then applied to train a regression ensemble. Although none of these three sets of measures is able to rep- resent the semantic similarity of two sentences individually, our experimental results show that the combination of these features can pre- cisely assess the semantic similarity of the sentences. In the English subtask our system’s best result ranked 35 among 73 system runs with 0.7189 average Pearson correlation over five test sets. This was 0.08 correlation points less than the best submitted run. 1 Introduction Semantic textual similarity (STS) aims to automat- ically estimate the relatedness of the meaning of sentences (Agirre et al., 2015). The literature con- sists of a series of well-established frameworks to explore a deeper understanding of the semantic relationship between entities, ranging from onto- logical reasoning to compositional as well as dis- tributional semantics (Cohen et al., 2009). However, automatically estimating the semantic similarity of full sentences is still a challenging task. Our system aims to quantify the similarity of pairs of sentences by encoding a variety of related- ness features in a vector of attributes and then pre- dicting their similarity scores by employing machine-learning algorithms. Different syntactic, semantic, and structural similarity measures have been applied to quantify the similarity of texts. We have chosen to approach the estimation of similari- ty as a regression problem. Hence, we use the quantified similarity of sentence pairs to train a regressor that can then be applied to predict simi- larity scores for the unseen pairs. The paper is structured as follows: Section 2 presents the pro- posed similarity measures. In Section 3, the regres- sion models are introduced and the experimental results are discussed in detail. The conclusions are summarized in Section 4. 2 Similarity Measures In this section we describe the similarity measures we have employed to calculate semantic related- ness of pairs of sentences. 2.1 Syntactic Similarity Measures Bags of words overlap: A simple measure for computing the similarity of a sentence pair is the number of words they have in common. Although a pair of sentences with the same bag of words (i.e. unordered list of all words of a sentence) can con- vey completely different meanings, this measure along with some structural measures can form an effective criterion for semantic comparison. Bags of lemmatised/stemmed words …

Content truncated to 500 words