Context Sensitive Neural Lemmatization with Lematus

Proceedings of NAACL-HLT 2018, pages 1391–1400 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Context Sensitive Neural Lemmatization with Lematus Toms Bergmanis School of Informatics University of Edinburgh T.Bergmanis@sms.ed.ac.uk Sharon Goldwater School of Informatics University of Edinburgh sgwater@inf.ed.ac.uk Abstract The main motivation for developing context- sensitive lemmatizers is to improve perfor- mance on unseen and ambiguous words. Yet previous systems have not carefully evaluated whether the use of context actually helps in these cases. We introduce Lematus, a lemma- tizer based on a standard encoder-decoder ar- chitecture, which incorporates character-level sentence context. We evaluate its lemmatiza- tion accuracy across 20 languages in both a full data setting and a lower-resource setting with 10k training examples in each language. In both settings, we show that including con- text significantly improves results against a context-free version of the model. Context helps more for ambiguous words than for un- seen words, though the latter has a greater effect on overall performance differences be- tween languages. We also compare to three previous context-sensitive lemmatization sys- tems, which all use pre-extracted edit trees as well as hand-selected features and/or addi- tional sources of information such as tagged training data. Without using any of these, our context-sensitive model outperforms the best competitor system (Lemming) in the full- data setting, and performs on par in the lower- resource setting. 1 Introduction Lemmatization is the process of determining the dictionary form of a word (e.g. swim) given one of its inflected variants (e.g. swims, swimming, swam, swum). Data-driven lemmatizers face two main challenges: first, to generalize beyond the training data in order to lemmatize unseen words; and sec- ond, to disambiguate ambiguous wordforms from their sentence context. In Latvian, for example, the wordform “cel¸u” is ambiguous when considered in isolation: it could be an inflected variant of the verb “celt” (to lift) or the nouns “celis” (knee) or “cel¸ˇs” (road); without context, the lemmatizer can only guess. By definition, sentence context (or latent infor- mation derived from it, such as the target word’s morphosyntactic tags) is needed in order to cor- rectly lemmatize ambiguous forms such as the example above. Previous researchers have also assumed that context should help in lemmatiz- ing unseen words (Chrupała, 2006; M¨uller et al., 2015)—i.e., that the context contains useful fea- tures above and beyond those in the wordform it- self. Nevertheless, we are not aware of any previ- ous work that has attempted to quantify how much (or even whether) context actually helps in both of these cases. Several previous papers on context- sensitive lemmatization have reported results on unseen words (Chrupała, 2006; Chrupała et al., 2008; M¨uller et al., 2015; Chakrabarty et al., 2017), and some have compared versions of their systems that use context in different ways (M¨uller et al., 2015; Chakrabarty et al., 2017), but there are few if any direct comparisons between context-sensitive and context-free systems, nor have results been reported on ambiguous forms. This paper presents Lematus—a system that adapts the neural …

Content truncated to 500 words