Contextualization of Morphological Inflection

Proceedings of NAACL-HLT 2019, pages 2018–2024 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2018 Contextualization of Morphological Inflection Ekaterina Vylomova@ Ryan CotterellP,H Timothy Baldwin@ Trevor Cohn@ and Jason EisnerP @School of Computing and Information Systems, University of Melbourne, Melbourne, Australia HThe Computer Laboratory, University of Cambridge, Cambridge, UK PDepartment of Computer Science, Johns Hopkins University, Baltimore, USA {vylomovae,tbaldwin,tcohn}@unimelb.edu.au rdc42@cam.ac.uk jason@cs.jhu.edu Abstract Critical to natural language generation is the production of correctly inflected text. In this paper, we isolate the task of predicting a fully inflected sentence from its partially lemma- tized version. Unlike traditional morphologi- cal inflection or surface realization, our task input does not provide “gold” tags that spec- ify what morphological features to realize on each lemmatized word; rather, such features must be inferred from sentential context. We develop a neural hybrid graphical model that explicitly reconstructs morphological features before predicting the inflected forms, and com- pare this to a system that directly predicts the inflected forms without relying on any morphological annotation. We experiment on several typologically diverse languages from the Universal Dependencies treebanks, show- ing the utility of incorporating linguistically- motivated latent variables into NLP models. 1 Introduction NLP systems are often required to generate grammatical text, e.g., in machine translation, summarization, dialogue, and grammar correction. One component of grammaticality is the use of contextually appropriate closed-class morphemes. In this work, we study contextual inflection, which has been recently introduced in the CoNLL- SIGMORPHON 2018 shared task (Cotterell et al., 2018) to directly investigate context-dependent morphology in NLP. There, a system must inflect partially lemmatized tokens in sentential context. For example, in English, the system must recon- struct the correct word sequence two cats are sitting from partially lemmatized sequence two _cat_ are sitting. Among other things, this requires: (1) identifying cat as a noun in this context, (2) recog- nizing that cat should be inflected as plural to agree with the nearby verb and numeral, and (3) realizing this inflection as the suffix s. Most past work in supervised computational morphology—including the previous CoNLL-SIGMORPHON shared tasks on morphological reinflection (Cotterell et al., 2017)—has focused mainly on step (3) above. As the task has been introduced into the literature only recently, we provide some background. Con- textual inflection amounts to a highly constrained version of language modeling. Language modeling predicts all words of a sentence from scratch, so the usual training and evaluation metric— perplexity—is dominated by the language model’s ability to predict content, which is where most of the uncertainty lies. Our task focuses on just the ability to reconstruct certain missing parts of the sentence—inflectional morphemes and their ortho- graphic realization. This refocuses the modeling effort from semantic coherence to morphosyntactic coherence, an aspect of language that may take a back seat in current language models (see Linzen et al., 2016; Belinkov et al., 2017). Contextual in- flection does not perfectly separate grammaticality inmodeling from content modeling: as illustrated Tab. 1, mapping two cats _be_ sitting the fully-inflected …

Content truncated to 500 words