Dirichlet Processes for Joint Learning of Morphology and PoS Tags

International Joint Conference on Natural Language Processing, pages 1087–1091, Nagoya, Japan, 14-18 October 2013. Dirichlet Processes for Joint Learning of Morphology and PoS Tags Burcu Can Department of Computer Engineering Hacettepe University Beytepe, Ankara 06800 Turkey burcu.can@hacettepe.edu.tr Suresh Manandhar Department of Computer Science University of York Heslington, York, YO10 5GH, UK suresh.manandhar@york.ac.uk Abstract This paper presents a joint model for learning morphology and part-of-speech (PoS) tags simultaneously. The proposed method adopts a finite mixture model that groups words having similar contextual features thereby assigning the same PoS tag to those words. While learning PoS tags, words are analysed morphologically by exploiting similar morphological fea- tures of the learned PoS tags. The results show that morphology and PoS tags can be learned jointly in a fully unsupervised setting. 1 Introduction The morphology of a word is an important indica- tor that determines its PoS tag, meanwhile the PoS tag of a word helps in identifying the correct mor- phological segmentation of the word. This rela- tionship between morphology and syntax has been beneficial in both morphology learning with the exploitation of the syntactic features and in PoS tagging with the adoption of morphological fea- tures. There has been a number of research that have performed PoS tagging by making use of morpho- logical information (Clark (2003), Hasan and Ng (2009), Abend et al. (2010), Christodoulopoulos et al. (2011), etc.). There has been also a number of other research that have performed morpholog- ical segmentation by adopting syntactic informa- tion (Hu et al. (2005), Can and Manandhar (2009), Lee et al. (2011), etc.). However, there is a small number of research that combines two tasks in a single framework. Sirts and Alum¨ae (2012) share a similar goal with us in joining PoS tagging and morphologi- cal segmentation in a single framework. They use hierarchical Dirichlet process for infinite HMMs to induce both PoS tags and morphological seg- mentation. Their model is type-based, whereas our model is token based. In our model, we use finite mixture models for PoS tagging and Dirich- let processes for segmentation. 2 Model Definition The generative story of the model goes as follows: 1. Draw a PoS tag ci. 2. Generate a word wi that belongs to ci. 3. Generate the context ci−1,i+1 of the word wi from ci. 4. From the possible splits of wi, generate a suffix mi conditioned on ci, such that wi = si + mi, where si denotes the stem. The generative story is summarised as follows: p(ci, ci−1,i+1, wi, s, m) = p(ci)p(ci−1,i+1|ci) p(wi|ci)p(m|ci)p(s) 2.1 PoS Tagging The model adopts a finite mixture model for PoS tagging (see Figure 1). Each mixture component represents a PoS tag that shares a set of features with other members in the same component. Each mixture component ci consists of 1. a distribution over contexts and 2. a distribution over words. Each context is a PoS tag pair < ci−1, ci+1 > where the previous word wi−1 belongs to ci−1 and the following word wi+1 belongs …

Content truncated to 500 words