A Hierarchical Dirichlet Process Model for Joint Part-of-Speech and Morphology Induction

2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 407–416, Montr´eal, Canada, June 3-8, 2012. c⃝2012 Association for Computational Linguistics A Hierarchical Dirichlet Process Model for Joint Part-of-Speech and Morphology Induction Kairit Sirts Institute of Cybernetics at Tallinn University of Technology kairit.sirts@phon.ioc.ee Tanel Alum¨ae Institute of Cybernetics at Tallinn University of Technology tanel.alumae@phon.ioc.ee Abstract In this paper we present a fully unsupervised nonparametric Bayesian model that jointly in- duces POS tags and morphological segmen- tations. The model is essentially an infi- nite HMM that infers the number of states from data. Incorporating segmentation into the same model provides the morphological features to the system and eliminates the need to find them during preprocessing step. We show that learning both tasks jointly actually leads to better results than learning either task with gold standard data from the other task provided. The evaluation on multilingual data shows that the model produces state-of-the-art results on POS induction. 1 Introduction Nonparametric Bayesian modeling has recently be- come very popular in natural language processing (NLP), mostly because of its ability to provide pri- ors that are especially suitable for tasks in NLP (Teh, 2006). Using nonparametric priors enables to treat the size of the model as a random variable with its value to be induced during inference which makes its use very appealing in models that need to decide upon the number of states. The task of unsupervised parts-of-speech (POS) tagging has been under research in numerous pa- pers, for overview see (Christodoulopoulos et al., 2010). Most of the POS induction models use the structure of hidden Markov model (HMM) (Rabiner, 1989) that requires the knowledge about the num- ber of hidden states (corresponding to the number of tags) in advance. According to our consider- ations, supplying this information is not desirable for two opposing reasons: 1) it injects into the sys- tem a piece of knowledge which in a truly unsu- pervised setting would be unavailable; and 2) the number of POS tags used is somewhat arbitrary any- way because there is no common consensus of what should be the true number of tags in each language and therefore it seems unreasonable to constrain the model with such a number instead of learning it from the data. Unsupervised morphology learning is another popular task that has been extensively studied by many authors. Here we are interested in learning concatenative morphology of words, meaning the substrings of the word corresponding to morphemes that, when concatenated, will give the lexical repre- sentation of the word type. For the rest of the paper we will refer to this task as (morphological) segmen- tation. Several unsupervised POS induction systems make use of morphological features (Blunsom and Cohn, 2011; Lee et al., 2010; Berg-Kirkpatrick et al., 2010; Clark, 2003; Christodoulopoulos et al., 2011) and this approach has been empirically proved to be helpful (Christodoulopoulos et al., 2010). In a similar fashion one could think that knowing POS tags could be useful …

Content truncated to 500 words