Reverse-engineering Language: A Study on the Semantic Compositionality of German Compounds

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1637–1642, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Reverse-engineering Language: A Study on the Semantic Compositionality of German Compounds Corina Dima Collaborative Research Center 833 University of T¨ubingen, Germany corina.dima@uni-tuebingen.de Abstract In this paper we analyze the perfor- mance of different composition models on a large dataset of German compound nouns. Given a vector space model for the German language, we try to reconstruct the observed representation (the corpus- estimated vector) of a compound by com- posing the observed representations of its two immediate constituents. We explore the composition models proposed in the literature and also present a new, simple model that achieves the best performance on our dataset. 1 Introduction Vector space models of language like the ones presented in (Collobert et al., 2011b; Mikolov et al., 2013; Pennington et al., 2014) create good representations for the individual words of a lan- guage. However, the words in a language can be combined into infinitely many distinct, well- formed phrases and sentences. Creating meaning- ful, reusable representations for such longer word sequences is still an open problem. In this paper we focus on building represen- tations for syntactic units just above the word level, by exploring compositional models for com- pounds. Bauer (2001) defines a compound as “a lexical unit made up of two or more elements, each of which can function as a lexeme indepen- dent of the other(s) in other contexts” (e.g. apple tree). The vast majority of compounds are com- positional, i.e. we can understand the meaning of the compound if we know the meaning of its con- stituent words. We would like to equip the vector space model with a composition function able to construct a composite representation for apple tree from the representations of apple and tree. The composite representation should ideally be indis- tinguishable from its observed representation, i.e. the representation learned directly by the language model if the compound is part of the dictionary. We situate our investigations in the context of the German language, a language where com- pounds represent an important fraction of the vo- cabulary. Baroni et al. (2002) analyzed the 28 mil- lion words German APA news corpus and discov- ered that compounds account for 47% of the word types but only 7% of the overall token count, with 83% of compounds having a corpus frequency of 5 or lower. The high productivity of the compound- ing process makes the compositional approach the most tractable way to create meaningful represen- tations for all the compounds that have been or will be coined by the speakers of the German language. German compounds have a strategic advantage for our study: they are generally written as a contiguous word, irrespective of how many con- stituents they have. Our example English com- pound, apple tree, translates into the German com- pound Apfelbaum, with the head Baum “tree” and the modifier Apfel “apple”. Because the com- pound is …

Content truncated to 500 words