Retrofitting Sense-Specific Word Vectors Using Parallel Text

Proceedings of NAACL-HLT 2016, pages 1378–1383, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Retrofitting Sense-Specific Word Vectors Using Parallel Text Allyson Ettinger1, Philip Resnik1,3, Marine Carpuat2,3 1Linguistics, 2Computer Science, 3Institute for Advanced Computer Studies University of Maryland, College Park, MD {aetting, resnik}@umd.edu, marine@cs.umd.edu Abstract Jauhar et al. (2015) recently proposed to learn sense-specific word representations by “retrofitting” standard distributional word rep- resentations to an existing ontology. We ob- serve that this approach does not require an ontology, and can be generalized to any graph defining word senses and relations between them. We create such a graph using transla- tions learned from parallel corpora. On a set of lexical semantic tasks, representations learned using parallel text perform roughly as well as those derived from WordNet, and combining the two representation types significantly im- proves performance. 1 Introduction Vector space models (VSMs) provide a powerful tool for representing word meanings and modeling the relations between them. While these models have demonstrated impressive success in capturing some aspects of word meaning (Landauer and Du- mais, 1997; Turney et al., 2010; Mikolov et al., 2013; Baroni et al., 2014; Levy et al., 2014), they generally fail to capture the fact that single word forms often have multiple meanings. This can lead to counterintuitive results—for example, it should be possible for the nearest word to rock to be stone in everyday usage, punk in discussions of music, and crack (cocaine) in discussions about drugs. In a recent paper, Jauhar et al. (2015) introduce a method for “retrofitting” generic word vectors to create sense-specific vectors using the WordNet se- mantic lexicon (Miller, 1995). From WordNet, they create a graph structure comprising two classes of relations: form-based relations between each word form and its respective senses, and meaning-based relations between word senses with similar mean- ings. This graph structure is then used to transform a traditional VSM into an enriched VSM, where each point in the space represents a word sense, rather than a word form. This approach is appealing as, un- like with prior sense-aware representations, senses are defined categories in a semantic lexicon, rather than clusters induced from raw text (Reisinger and Mooney, 2010; Huang et al., 2012; Neelakantan et al., 2015; Tian et al., 2014), and the method does not require performing word sense disambiguation (Guo et al., 2014). In this paper, we observe that the crucial mean- ing relationships in the Jauhar et al. retrofitting process—the word sense graph—can be inferred based on another widely available resource: bilin- gual parallel text. This observation is grounded in a well-established tradition of using cross-language correspondences as a form of sense annotation (Gale et al., 1992; Diab and Resnik, 2002; Ng et al., 2003; Carpuat and Wu, 2007; Lefever and Hoste, 2010, and others). Using parallel text to define sense dis- tinctions sidesteps the persistent difficulty of identi- fying a single correct sense partitioning based on hu- man intuition, and avoids large investments in man- ual curation or annotation. We use parallel …

Content truncated to 500 words