DLS@CU: Sentence Similarity from Word Alignment

Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 241–246, Dublin, Ireland, August 23-24, 2014. DLS@CU: Sentence Similarity from Word Alignment Md Arafat Sultan†, Steven Bethard‡, Tamara Sumner† †Institute of Cognitive Science and Department of Computer Science University of Colorado Boulder ‡Department of Computer and Information Sciences University of Alabama at Birmingham arafat.sultan@colorado.edu, bethard@cis.uab.edu, sumner@colorado.edu Abstract We present an algorithm for computing the semantic similarity between two sen- tences. It adopts the hypothesis that se- mantic similarity is a monotonically in- creasing function of the degree to which (1) the two sentences contain similar se- mantic units, and (2) such units occur in similar semantic contexts. With a simplis- tic operationalization of the notion of se- mantic units with individual words, we ex- perimentally show that this hypothesis can lead to state-of-the-art results for sentence- level semantic similarity. At the Sem- Eval 2014 STS task (task 10), our system demonstrated the best performance (mea- sured by correlation with human annota- tions) among 38 system runs. 1 Introduction Semantic textual similarity (STS), in the context of short text fragments, has drawn considerable attention in recent times. Its application spans a multitude of areas, including natural language pro- cessing, information retrieval and digital learning. Examples of tasks that benefit from STS include text summarization, machine translation, question answering, short answer scoring, and so on. The annual series of SemEval STS tasks (Agirre et al., 2012; Agirre et al., 2013; Agirre et al., 2014) is an important platform where STS systems are evaluated on common data and evaluation criteria. In this article, we describe an STS system which participated and outperformed all other systems at SemEval 2014. The algorithm is a straightforward application of the monolingual word aligner presented in (Sul- This work is licensed under a Creative Commons At- tribution 4.0 International Licence. Page numbers and pro- ceedings footer are added by the organisers. Licence details: http://creativecommons.org/licenses/by/4.0/ tan et al., 2014). This aligner aligns related words in two sentences based on the following properties of the words: 1. They are semantically similar. 2. They occur in similar semantic contexts in the respective sentences. The output of the word aligner for a sentence pair can be used to predict the pair’s semantic sim- ilarity by taking the proportion of their aligned content words. Intuitively, the more semantic components in the sentences we can meaningfully align, the higher their semantic similarity should be. In experiments on STS 2013 data reported by Sultan et al. (2014), this approach was found highly effective. We also adopt this hypothesis of semantic compositionality for STS 2014. We implement an STS algorithm that is only slightly different from the algorithm in (Sultan et al., 2014). The approach remains equally success- ful on STS 2014 data. 2 Background We focus on two relevant topics in this section: the state of the art of STS research, and the word aligner presented in (Sultan et al., 2014). 2.1 Semantic Textual Similarity Since the inception of textual similarity research for …

Content truncated to 500 words