Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text

Proceedings of NAACL-HLT 2019, pages 4119–4128 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 4119 Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Toms Bergmanis School of Informatics University of Edinburgh T.Bergmanis@sms.ed.ac.uk Sharon Goldwater School of Informatics University of Edinburgh sgwater@inf.ed.ac.uk Abstract Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Using context can help, both for unseen and ambiguous words. Yet most context-sensitive approaches require full lemma-annotated sentences for training, which may be scarce or unavailable in low- resource languages. In addition (as shown here), in a low-resource setting, a lemmatizer can learn more from n labeled examples of distinct words (types) than from n (contigu- ous) labeled tokens, since the latter contain far fewer distinct types. To combine the ef- ficiency of type-based learning with the ben- efits of context, we propose a way to train a context-sensitive lemmatizer with little or no labeled corpus data, using inflection tables from the UniMorph project and raw text exam- ples from Wikipedia that provide sentence con- texts for the unambiguous UniMorph exam- ples. Despite these being unambiguous exam- ples, the model successfully generalizes from them, leading to improved results (both over- all, and especially on unseen words) in com- parison to a baseline that does not use context. 1 Introduction Many lemmatizers work on isolated wordforms (Wicentowski, 2002; Dreyer et al., 2008; Rastogi et al., 2016; Makarov and Clematide, 2018b,a). Lemmatizing in context can improve accuracy on ambiguous and unseen words (Bergmanis and Goldwater, 2018), but most systems for context- sensitive lemmatization must train on complete sen- tences labeled with POS and/or morphological tags as well as lemmas, and have only been tested with 20k-300k training tokens (Chrupała et al., 2008; M¨uller et al., 2015; Chakrabarty et al., 2017).1 1The smallest of these corpora contains 20k tokens of Bengali annotated only with lemmas, which Chakrabarty et al. (2017) reported took around two person months to create. Intuitively, though, sentence-annotated data is inefficient for training a lemmatizer, especially in low-resource settings. Training on (say) 1000 word types will provide far more information about a language’s morphology than training on 1000 con- tiguous tokens, where fewer types are represented. As noted above, sentence data can help with am- biguous and unseen words, but we show here that when data is scarce, this effect is small relative to the benefit of seeing more word types.2 Mo- tivated by this result, we propose a training data augmentation method that combines the efficiency of type-based learning and the expressive power of a context-sensitive model.3 We use Lematus (Bergmanis and Goldwater, 2018), a state-of-the- art lemmatizer that learns from lemma-annotated words in their N-character contexts. No predic- tions about surrounding words are used, so fully annotated training sentences are not needed. We exploit this fact by combining two sources of train- ing data: 1k lemma-annotated types (with con- texts) from the Universal Dependency Treebank (UDT) v2.24 …

Content truncated to 500 words