How to make words with vectors: Phrase generation in distributional semantics

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 624–633, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics How to make words with vectors: Phrase generation in distributional semantics Georgiana Dinu and Marco Baroni Center for Mind/Brain Sciences University of Trento, Italy (georgiana.dinu|marco.baroni)@unitn.it Abstract We introduce the problem of generation in distributional semantics: Given a distri- butional vector representing some mean- ing, how can we generate the phrase that best expresses that meaning? We mo- tivate this novel challenge on theoretical and practical grounds and propose a sim- ple data-driven approach to the estimation of generation functions. We test this in a monolingual scenario (paraphrase gen- eration) as well as in a cross-lingual set- ting (translation by synthesizing adjective- noun phrase vectors in English and gener- ating the equivalent expressions in Italian). 1 Introduction Distributional methods for semantics approximate the meaning of linguistic expressions with vectors that summarize the contexts in which they occur in large samples of text. This has been a very suc- cessful approach to lexical semantics (Erk, 2012), where semantic relatedness is assessed by compar- ing vectors. Recently these methods have been extended to phrases and sentences by means of composition operations (see Baroni (2013) for an overview). For example, given the vectors repre- senting red and car, composition derives a vector that approximates the meaning of red car. However, the link between language and mean- ing is, obviously, bidirectional: As message recip- ients we are exposed to a linguistic expression and we must compute its meaning (the synthesis prob- lem). As message producers we start from the meaning we want to communicate (a “thought”) and we must encode it into a word sequence (the generation problem). If distributional semantics is to be considered a proper semantic theory, then it must deal not only with synthesis (going from words to vectors), but also with generation (from vectors to words). Besides these theoretical considerations, phrase generation from vectors has many useful applica- tions. We can, for example, synthesize the vector representing the meaning of a phrase or sentence, and then generate alternative phrases or sentences from this vector to accomplish true paraphrase generation (as opposed to paraphrase detection or ranking of candidate paraphrases). Generation can be even more useful when the source vector comes from another modality or lan- guage. Recent work on grounding language in vi- sion shows that it is possible to represent images and linguistic expressions in a common vector- based semantic space (Frome et al., 2013; Socher et al., 2013). Given a vector representing an im- age, generation can be used to productively con- struct phrases or sentences that describe the im- age (as opposed to simply retrieving an existing description from a set of candidates). Translation is another potential application of the generation framework: Given a semantic space shared be- tween two or more languages, one can compose a word sequence in one language and generate trans- lations in another, with the shared semantic vector …

Content truncated to 500 words