Distributional Semantics Beyond Words: Supervised Learning of Analogy and Paraphrase

Distributional Semantics Beyond Words: Supervised Learning of Analogy and Paraphrase Peter D. Turney National Research Council Canada Information and Communications Technologies Ottawa, Ontario, Canada, K1A 0R6 peter.turney@nrc-cnrc.gc.ca Abstract There have been several efforts to extend distributional semantics beyond individual words, to measure the similarity of word pairs, phrases, and sentences (briefly, tuples; ordered sets of words, contiguous or noncontiguous). One way to extend beyond words is to com- pare two tuples using a function that com- bines pairwise similarities between the com- ponent words in the tuples. A strength of this approach is that it works with both rela- tional similarity (analogy) and compositional similarity (paraphrase). However, past work required hand-coding the combination func- tion for different tasks. The main contribution of this paper is that combination functions are generated by supervised learning. We achieve state-of-the-art results in measuring relational similarity between word pairs (SAT analo- gies and SemEval 2012 Task 2) and measur- ing compositional similarity between noun- modifier phrases and unigrams (multiple- choice paraphrase questions). 1 Introduction Harris (1954) and Firth (1957) hypothesized that words that appear in similar contexts tend to have similar meanings. This hypothesis is the founda- tion for distributional semantics, in which words are represented by context vectors. The similarity of two words is calculated by comparing the two cor- responding context vectors (Lund et al., 1995; Lan- dauer and Dumais, 1997; Turney and Pantel, 2010). Distributional semantics is highly effective for measuring the semantic similarity between individ- ual words. On a set of eighty multiple-choice syn- onym questions from the test of English as a for- eign language (TOEFL), a distributional approach recently achieved 100% accuracy (Bullinaria and Levy, 2012). However, it has been difficult to extend distributional semantics beyond individual words, to word pairs, phrases, and sentences. Moving beyond individual words, there are vari- ous types of semantic similarity to consider. Here we focus on paraphrase and analogy. Paraphrase is similarity in the meaning of two pieces of text (Androutsopoulos and Malakasiotis, 2010). Anal- ogy is similarity in the semantic relations of two sets of words (Turney, 2008a). It is common to study paraphrase at the sentence level (Androutsopoulos and Malakasiotis, 2010), but we prefer to concentrate on the simplest type of paraphrase, where a bigram paraphrases a unigram. For example, dog house is a paraphrase of kennel. In our experiments, we concentrate on noun-modifier bigrams and noun unigrams. Analogies map terms in one domain to terms in another domain (Gentner, 1983). The familiar anal- ogy between the solar system and the Rutherford- Bohr atomic model involves several terms from the domain of the solar system and the domain of the atomic model (Turney, 2008a). The simplest type of analogy is proportional anal- ogy, which involves two pairs of words (Turney, 2006b). For example, the pair ⟨cook, raw⟩ is anal- ogous to the pair ⟨decorate, plain⟩. If we cook a thing, it is no longer raw; if we decorate a thing, it 353 Transactions of the Association for Computational Linguistics, 1 (2013) …

Content truncated to 500 words