Improving Lemmatization of Non-Standard Languages with Joint Learning

Proceedings of NAACL-HLT 2019, pages 1493–1503 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1493 Improving Lemmatization of Non-Standard Languages with Joint Learning Enrique Manjavacas1, ´Akos K´ad´ar2, and Mike Kestemont1 1Computational Linguistics and Psycholinguistics Research Center, University of Antwerp, Belgium 2Tilburg Center for Cognition and Communication, Tilburg University, The Netherlands {firstname,lastname}@uantwerpen.be,a.kadar@uvt.nl Abstract Lemmatization of standard languages is con- cerned with (i) abstracting over morphologi- cal differences and (ii) resolving token-lemma ambiguities of inflected words in order to map them to a dictionary headword. In the present paper we aim to improve lemmatization per- formance on a set of non-standard histori- cal languages in which the difficulty is in- creased by an additional aspect (iii): spelling variation due to lacking orthographic stan- dards. We approach lemmatization as a string- transduction task with an encoder-decoder ar- chitecture which we enrich with sentence con- text information using a hierarchical sentence encoder. We show significant improvements over the state-of-the-art when training the sen- tence encoder jointly for lemmatization and language modeling. Crucially, our architec- ture does not require POS or morphological annotations, which are not always available for historical corpora. Additionally, we also test the proposed model on a set of typolog- ically diverse standard languages showing re- sults on par or better than a model without en- hanced sentence representations and previous state-of-the-art systems. Finally, to encourage future work on processing of non-standard va- rieties, we release the dataset of non-standard languages underlying the present study, based on openly accessible sources. 1 Introduction Lemmatization is the task of mapping a token to its corresponding dictionary head-form to allow downstream applications to abstract away from orthographic and inflectional variation (Knowles and Mohd Don, 2004). While lemmatization is considered to be solved for analytic and resource- rich languages such as English, it remains an open challenge for morphologically complex (e.g. Es- tonian, Latvian) and low-resource languages with unstable orthography (e.g. historical languages). Especially for languages with higher surface vari- ation, lemmatization plays a crucial role as a pre- processing step for downstream tasks such as topic modeling, stylometry and information retrieval. In the case of standard languages, lemmatiza- tion complexity arises primarily from two sources: (i) morphological complexity affecting the num- ber of inflectional patterns a lemmatizer has to model and (ii) token-lemma ambiguities (e.g. “liv- ing” can refer to lemmas “living” or “live”) which require modeling sentence context information. In the case of historical languages, however, the aforementioned spelling variation introduces fur- ther complications. For instance, the regularity of the morphological system is drastically reduced since the evidence supporting token-lemma map- pings becomes more sparse. As an example, while the modern Dutch lemma “jaar” (en. “year”) can be inflected in 2 different ways (“jaar”, “jaren”), in a Middle Dutch corpus used in this study it is found in combination with 70 different forms (“iare”, “ior”, “jaer”, etc.). Moreover, spelling variation increases token-lemma ambiguities by conflating surface realizations of otherwise unam- biguous tokens—e.g. Middle Low German “bath” can refer to …

Content truncated to 500 words