Discriminative Improvements to Distributional Sentence Similarity

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 891–896, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Discriminative Improvements to Distributional Sentence Similarity Yangfeng Ji School of Interactive Computing Georgia Institute of Technology jiyfeng@gatech.edu Jacob Eisenstein School of Interactive Computing Georgia Institute of Technology jacobe@gatech.edu Abstract Matrix and tensor factorization have been ap- plied to a number of semantic relatedness tasks, including paraphrase identification. The key idea is that similarity in the latent space implies semantic relatedness. We describe three ways in which labeled data can im- prove the accuracy of these approaches on paraphrase classification. First, we design a new discriminative term-weighting metric called TF-KLD, which outperforms TF-IDF. Next, we show that using the latent repre- sentation from matrix factorization as features in a classification algorithm substantially im- proves accuracy. Finally, we combine latent features with fine-grained n-gram overlap fea- tures, yielding performance that is 3% more accurate than the prior state-of-the-art. 1 Introduction Measuring the semantic similarity of short units of text is fundamental to many natural language processing tasks, from evaluating machine transla- tion (Kauchak and Barzilay, 2006) to grouping re- dundant event mentions in social media (Petrovi´c et al., 2010). The task is challenging because of the infinitely diverse set of possible linguistic real- izations for any idea (Bhagat and Hovy, 2013), and because of the short length of individual sentences, which means that standard bag-of-words representa- tions will be hopelessly sparse. Distributional methods address this problem by transforming the high-dimensional bag-of-words representation into a lower-dimensional latent space. This can be accomplished by factoring a matrix or tensor of term-context counts (Turney and Pan- tel, 2010); proximity in the induced latent space has been shown to correlate with semantic similar- ity (Mihalcea et al., 2006). However, factoring the term-context matrix means throwing away a consid- erable amount of information, as the original ma- trix of size M × N (number of instances by number of features) is factored into two smaller matrices of size M × K and N × K, with K ≪ M, N. If the factorization does not take into account labeled data about semantic similarity, important information can be lost. In this paper, we show how labeled data can con- siderably improve distributional methods for mea- suring semantic similarity. First, we develop a new discriminative term-weighting metric called TF-KLD, which is applied to the term-context ma- trix before factorization. On a standard paraphrase identification task (Dolan et al., 2004), this method improves on both traditional TF-IDF and Weighted Textual Matrix Factorization (WTMF; Guo and Diab, 2012). Next, we convert the latent repre- sentations of each sentence pair into a feature vec- tor, which is used as input to a linear SVM clas- sifier. This yields further improvements and sub- stantially outperforms the current state-of-the-art on paraphrase classification. We then add “fine- grained” features about the lexical similarity of the sentence pair. The combination of latent and fine- grained features yields further improvements in ac- …

Content truncated to 500 words