Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics

Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 1517–1526, Sofia, Bulgaria, August 4-9 2013. c⃝2013 Association for Computational Linguistics Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics Angeliki Lazaridou and Marco Marelli and Roberto Zamparelli and Marco Baroni Center for Mind/Brain Sciences (University of Trento, Italy) first.last@unitn.it Abstract Speakers of a language can construct an unlimited number of new words through morphological derivation. This is a major cause of data sparseness for corpus-based approaches to lexical semantics, such as distributional semantic models of word meaning. We adapt compositional meth- ods originally developed for phrases to the task of deriving the distributional meaning of morphologically complex words from their parts. Semantic representations con- structed in this way beat a strong baseline and can be of higher quality than represen- tations directly constructed from corpus data. Our results constitute a novel evalua- tion of the proposed composition methods, in which the full additive model achieves the best performance, and demonstrate the usefulness of a compositional morphology component in distributional semantics. 1 Introduction Effective ways to represent word meaning are needed in many branches of natural language pro- cessing. In the last decades, corpus-based meth- ods have achieved some degree of success in mod- eling lexical semantics. Distributional semantic models (DSMs) in particular represent the mean- ing of a word by a vector, the dimensions of which encode corpus-extracted co-occurrence statistics, under the assumption that words that are semanti- cally similar will occur in similar contexts (Turney and Pantel, 2010). Reliable distributional vectors can only be extracted for words that occur in many contexts in the corpus. Not surprisingly, there is a strong correlation between word frequency and vector quality (Bullinaria and Levy, 2007), and since most words occur only once even in very large corpora (Baroni, 2009), DSMs suffer data sparseness. While word rarity has many sources, one of the most common and systematic ones is the high pro- ductivity of morphological derivation processes, whereby an unlimited number of new words can be constructed by adding affixes to existing stems (Baayen, 2005; Bauer, 2001; Plag, 1999).1 For example, in the multi-billion-word corpus we in- troduce below, perfectly reasonable derived forms such as lexicalizable or affixless never occur. Even without considering the theoretically infinite num- ber of possible derived nonce words, and restrict- ing ourselves instead to words that are already listed in dictionaries, complex forms cover a high portion of the lexicon. For example, morphologi- cally complex forms account for 55% of the lem- mas in the CELEX English database (see Section 4.1 below). In most of these cases (80% according to our corpus) the stem is more frequent than the complex form (e.g., the stem build occurs 15 times more often than the derived form rebuild, and the latter is certainly not an unusual derived form). DSMs ignore derivational morphology alto- gether. Consequently, they cannot provide mean- ing representations for new derived forms, nor can they harness the systematic relation existing be- tween stems …

Content truncated to 500 words