Morphological disambiguation from stemming data

Proceedings of the 28th International Conference on Computational Linguistics, pages 4649–4660 Barcelona, Spain (Online), December 8-13, 2020 4649 Morphological disambiguation from stemming data Antoine Nzeyimana* University of Massachusetts, Amherst, MA 01003, USA anthonzeyi@gmail.com Abstract Morphological analysis and disambiguation is an important task and a crucial preprocessing step in natural language processing of morphologically rich languages. Kinyarwanda, a morpholog- ically rich language, currently lacks tools for automated morphological analysis. While linguis- tically curated finite state tools can be easily developed for morphological analysis, the mor- phological richness of the language allows many ambiguous analyses to be produced, requiring effective disambiguation. In this paper, we propose learning to morphologically disambiguate Kinyarwanda verbal forms from a new stemming dataset collected through crowd-sourcing. Us- ing feature engineering and a feed-forward neural network based classifier, we achieve about 89% non-contextualized disambiguation accuracy. Our experiments reveal that inflectional properties of stems and morpheme association rules are the most discriminative features for disambiguation. 1 Introduction For morphologically rich languages, morphological analysis and disambiguation plays a critical role in most natural language processing (NLP) tasks. When inflections are generated by piecing together mul- tiple morphemes, a large and sparse vocabulary is produced, requiring tools to unpack the individual morphemes for downstream NLP tasks such information extraction and machine translation. A key char- acteristic of these languages is that morphemes often have specific meanings (often relating to properties of the words they form or referring to contextual entities) and their combination into words is mostly regular. Figure 1 shows typical morphological units contained in the word ’ntuzamwibeshyeho’ (Never underestimate him/her). While several morphologically rich languages such as Turkish, Arabic and Modern Hebrew already have mature tools for morphological segmentation (C¨oltekin, 2010) (C¸ ¨oltekin, 2014) (Itai and Segal, 2003) (Habash and Rambow, 2006), Kinyarwanda still lacks appropriate tools for the task. A key limita- tion in the effort is the need to have high quality datasets manually annotated by language experts. With limited funding opportunities, research on NLP for low resource languages lags behind recent advance- ments made for NLP on high resource languages. In this work, we leverage an easy to collect stemming dataset and transform it into a resource for morphological disambiguation. While the focus here is on Kinyarwanda verbal forms, the method can be applied to other morphologically rich languages. Collect- ing stemming data is much faster and less prone to errors than full morphological segmentations which require subtle linguistic knowledge. Through a maximum entropy model, we are able to combine morphological properties of stems with inflectional similarity information from word embeddings to accurately disambiguate candidate segmen- tations from a rule-based morphological analyzer. Our work here pertains to non-contextual verb-phrase disambiguation but is a key step towards contextual disambiguation. We believe that this work will allow rich morphology to be incorporated in new models for Kinyarwanda and improve downstream NLP tasks on the language. * Part of this work was done while the author was a graduate student at the University of Oregon, Eugene, OR 97403, USA …

Content truncated to 500 words