A Simple Joint Model for Improved Contextual Neural Lemmatization

Proceedings of NAACL-HLT 2019, pages 1517–1528 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 1517 A Simple Joint Model for Improved Contextual Neural Lemmatization Chaitanya Malaviya*,B and Shijie Wu*,@ and Ryan Cotterell@,H BAllen Institute for Artificial Intelligence @Department of Computer Science, Johns Hopkins University HDepartment of Computer Science and Technology, University of Cambridge chaitanyam@allenai.org, shijie.wu@jhu.edu, rdc42@cam.ac.uk Abstract English verbs have multiple forms. For in- stance, talk may also appear as talks, talked or talking, depending on the context. The NLP task of lemmatization seeks to map these di- verse forms back to a canonical one, known as the lemma. We present a simple joint neural model for lemmatization and morpho- logical tagging that achieves state-of-the-art results on 20 languages from the Universal Dependencies corpora. Our paper describes the model in addition to training and de- coding procedures. Error analysis indicates that joint morphological tagging and lemma- tization is especially helpful in low-resource lemmatization and languages that display a larger degree of morphological complexity. Code and pre-trained models are available at https://sigmorphon.github.io/ sharedtasks/2019/task2/. 1 Introduction Lemmatization is a core NLP task that involves a string-to-string transduction from an inflected word form to its citation form, known as the lemma. More concretely, consider the English sentence: The bulls are running in Pamplona. A lemma- tizer will seek to map each word to a form you may find in a dictionary—for instance, mapping running to run. This linguistic normalization is important in several downstream NLP applications, especially for highly inflected languages. Lemmati- zation has previously been shown to improve recall for information retrieval (Kanis and Skorkovská, 2010; Monz and De Rijke, 2001), to aid machine translation (Fraser et al., 2012; Chahuneau et al., 2013) and is a core part of modern parsing systems (Björkelund et al., 2010; Zeman et al., 2018). However, the task is quite nuanced as the proper choice of the lemma is context dependent. For * Equal contribution. Listing order is random. Figure 1: Our structured neural model shown as a hy- brid (directed-undirected) graphical model (Koller and Friedman, 2009). Notionally, the wi denote inflected word forms, the mi denote morphological tags and the ℓi denote lemmata. instance, in the sentence A running of the bulls took place in Pamplona, the word running is its own lemma, since, here, running is a noun rather than an inflected verb. Several counter-examples exist to this trend, as discussed in depth in Haspelmath and Sims (2013). Thus, a good lemmatizer must make use of some representation of each word’s sentential context. The research question in this work is, then, how do we design a lemmatization model that best extracts the morpho-syntax from the sentential context? Recent work (Bergmanis and Goldwater, 2018) has presented a system that directly summarizes the sentential context using a recurrent neural network to decide how to lemmatize. As Bergmanis and Goldwater (2018)’s system currently achieves state-of-the-art results, it must implicitly learn a contextual representation that encodes the neces- sary morpho-syntax, as such knowledge is …

Content truncated to 500 words