Joint Semantic Synthesis and Morphological Analysis of the Derived Word

Joint Semantic Synthesis and Morphological Analysis of the Derived Word Ryan Cotterell Department of Computer Science Johns Hopkins University ryan.cotterell@jhu.edu Hinrich Sch¨utze CIS LMU Munich inquiries@cislmu.org Abstract Much like sentences are composed of words, words themselves are composed of smaller units. For example, the English word questionably can be analyzed as question+able+ly. However, this structural decomposition of the word does not directly give us a semantic representation of the word’s meaning. Since morphology obeys the principle of compositionality, the semantics of the word can be systematically derived from the meaning of its parts. In this work, we propose a novel probabilistic model of word formation that captures both the analysis of a word w into its constituent segments and the synthesis of the meaning of w from the mean- ings of those segments. Our model jointly learns to segment words into morphemes and compose distributional semantic vectors of those morphemes. We experiment with the model on English CELEX data and German DErivBase (Zeller et al., 2013) data. We show that jointly modeling semantics increases both segmentation accuracy and morpheme F1 by between 3% and 5%. Additionally, we investigate different models of vector compo- sition, showing that recurrent neural networks yield an improvement over simple additive models. Finally, we study the degree to which the representations correspond to a linguist’s notion of morphological productivity. 1 Introduction In most languages, words decompose further into smaller units, termed morphemes. For example, the English word questionably can be analyzed as question+able+ly. This structural decomposition of the word, however, by itself is not a semantic rep- resentation of the word’s meaning;1 we further re- quire an account of how to synthesize the meaning from the decomposition. Fortunately, words—just like phrases—to a large extent obey the principle of compositionality: the semantics of the word can be systematically derived from the meaning of its parts.2 In this work, we propose a novel joint prob- abilistic model of word formation that captures both structural decomposition of a word w into its con- stituent segments and the synthesis of w’s meaning from the meaning of those segments. Morphological segmentation is a structured pre- diction task that seeks to break a word up into its constituent morphemes. The output segmentation has been shown to aid a diverse set of applications, such as automatic speech recognition (Afify et al., 2006), keyword spotting (Narasimhan et al., 2014), machine translation (Clifton and Sarkar, 2011) and parsing (Seeker and C¸ etino˘glu, 2015). In contrast to much of this prior work, we focus on supervised segmentation, i.e., we provide the model with gold segmentations during training time. Instead of sur- 1There are many different linguistic and computational theo- ries for interpreting the structural decomposition of a word. For example, un- often signifies negation and its effect on semantics can then be modeled by theories based on logic. This work ad- dresses the question of structural decomposition and semantic synthesis in the general framework of distributional semantics. 2Morphological research in theoretical and computational linguistics often focuses …

Content truncated to 500 words