Unsupervised Learning of Morphological Forests

Unsupervised Learning of Morphological Forests Jiaming Luo CSAIL, MIT j_luo@mit.edu Karthik Narasimhan CSAIL, MIT karthikn@mit.edu Regina Barzilay CSAIL, MIT regina@csail.mit.edu Abstract This paper focuses on unsupervised modeling of morphological families, collectively com- prising a forest over the language vocabulary. This formulation enables us to capture edge- wise properties reflecting single-step morpho- logical derivations, along with global distribu- tional properties of the entire forest. These global properties constrain the size of the af- fix set and encourage formation of tight mor- phological families. The resulting objective is solved using Integer Linear Programming (ILP) paired with contrastive estimation. We train the model by alternating between op- timizing the local log-linear model and the global ILP objective. We evaluate our sys- tem on three tasks: root detection, clustering of morphological families, and segmentation. Our experiments demonstrate that our model yields consistent gains in all three tasks com- pared with the best published results.1 1 Introduction The morphological study of a language inherently draws upon the existence of families of related words. All words within a family can be derived from a common root via a series of transformations, whether inflectional or derivational. Figure 1 de- picts one such family, originating from the word faith. This representation can benefit a range of applications, including segmentation, root detection and clustering of morphological families. 1Code is available at https://github.com/ j-luo93/MorphForest. Figure 1: An illustration of a single tree in a mor- phological forest. pre and suf represent prefixation and suffixation. Each edge has an associated proba- bility for the morphological change. Using graph terminology, a full morphological as- signment of the words in a language can be repre- sented as a forest.2 Valid forests of morphological families exhibit a number of well-known regulari- ties. At the global level, the number of roots is lim- ited, and only constitute a small fraction of the vo- cabulary. A similar constraint applies to the num- ber of possible affixes, shared across families. At the local edge level, we prefer derivations that fol- low regular orthographic patterns and preserve se- mantic relatedness. We hypothesize that enforcing these constraints as part of the forest induction pro- 2The correct mathematical term for the structure in Figure 1 is a directed 1-forest or functional graph. For simplicity, we shall use the terms forest and tree to refer to a directed 1-forest or a directed 1-tree because of the cycle at the root. 353 Transactions of the Association for Computational Linguistics, vol. 5, pp. 353–364, 2017. Action Editor: Eric Fosler-Lussier. Submission batch: 10/2016; Revision batch: 12/2016; Published 10/2017. c⃝2017 Association for Computational Linguistics. Distributed under a CC-BY 4.0 license. cess will allow us to accurately learn morphological structures in an unsupervised fashion. To test this hypothesis, we define an objective over the entire forest representation. The proposed objective is designed to maximize the likelihood of local derivations, while constraining the overall number of affixes and encouraging tighter morpho- logical families. We optimize this objective using Integer Linear Programming (ILP), which is com- monly employed …

Content truncated to 500 words