Semi-Supervised Induction of POS-Tag Lexicons with Tree Models

Proceedings of Recent Advances in Natural Language Processing, pages 507–515, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_060 507 Semi-Supervised Induction of POS-Tag Lexicons with Tree Models Maciej Janicki University of Leipzig NLP Group, Department of Computer Science Augustusplatz 10, 04109 Leipzig macjan@o2.pl Abstract We approach the problem of POS tagging of morphologically rich languages in a set- ting where only a small amount of la- beled training data is available. We show that a bigram HMM tagger benefits from re-training on a larger untagged text us- ing Baum-Welch estimation. Most im- portantly, this estimation can be signifi- cantly improved by pre-guessing tags for OOV words based on morphological cri- teria. We consider two models for this task: a character-based recurrent neural network, which guesses the tag from the string form of the word, and a recently proposed graph-based model of morpho- logical transformations. In the latter, the unknown POS tags can be modeled as la- tent variables in a way very similar to Hidden Markov Tree models and an ana- logue of the Forward-Backward algorithm can be formulated, which enables us to compute expected values over unknown taggings. We evaluate both the quality of the induced tag lexicon and its im- pact on the HMM’s tagging accuracy. In both tasks, the graph-based morphology model performs significantly better than the RNN predictor. This confirms the in- tuition that morphologically related words provide useful information about an un- known word’s POS tag. 1 Introduction Part-of-speech tagging is nowadays commonly thought of as a solved problem, with accuracy scores easily achieving 95% or more. However, such results are typically reported for English or other resource-rich European languages, for which large amounts of high-quality training data are available. Those languages also tend to have a simple morphology and utilize small to mid- sized tagsets. However, for many other languages, the reality is different: training data are expen- sive or not available, more fine-grained tagsets are needed and complex morphology accounts for large numbers of OOV words in corpora. (Straka and Strakov´a, 2017) present a contemporary eval- uation of state-of-the-art POS tagging for a very wide variety of languages. The scores for tagging with UPOS1 tagset lie below 90% for many lan- guages. The lack of sufficient amounts of training data is undoubtedly one of the main reasons for such results. In this paper, we attempt to improve the POS tagging in a setting where only a small amount of labeled training data is available, as well as a sig- nificantly larger corpus of unlabeled text. We train a bigram Hidden Markov Model on the labeled part of the corpus and subsequently apply Baum- Welch estimation on the unlabeled part. Addition- ally, we use the labeled part to train a morphology model in an unsupervised setting. The key idea is to extend the tagger’s lexicon with words occur- ring in the unlabeled part before the Baum-Welch estimation and to pre-guess their possible tags us- ing the morphology model. The choice of a bigram HMM for tagging …

Content truncated to 500 words