Analogy Models for Neural Word Inflection

Proceedings of the 28th International Conference on Computational Linguistics, pages 2861–2878 Barcelona, Spain (Online), December 8-13, 2020 2861 Analogy Models for Neural Word Inflection Ling Liu and Mans Hulden University of Colorado first.last@colorado.edu Abstract Analogy is assumed to be the cognitive mechanism speakers resort to in order to inflect an unknown form of a lexeme based on knowledge of other words in a language. In this pro- cess, an analogy is formed between word forms within an inflectional paradigm but also across paradigms. As neural network models for inflection are typically trained only on lemma-target form pairs, we propose three new ways to provide neural models with additional source forms to strengthen analogy-formation, and compare our methods to other approaches in the literature. We show that the proposed methods of providing a Transformer sequence-to-sequence model with additional analogy sources in the input are consistently effective, and improve upon recent state-of-the-art results on 46 languages, particularly in low-resource settings. We also propose a method to combine the analogy-motivated approach with data hallucination or augmentation. We find that the two approaches are complementary to each other and combining the two approaches is especially helpful when the training data is extremely limited. 1 Introduction Morphological tasks such as the task of morphological inflection generation have attracted great research interest in recent years. SIGMORPHON has organized annual shared tasks on morphological inflection in the past five years (Cotterell et al., 2016; Cotterell et al., 2017a; Cotterell et al., 2018; McCarthy et al., 2019; Vylomova et al., 2020). In the typical SIGMORPHON shared task of morphological inflection, a lemma (citation form) and a morphosyntactic description (MSD) consisting of a set of features are provided, and the task is to generate an inflected form for the lemma corresponding to the MSD. Neural network models have been very successful in handling natural language processing (NLP) problems, and have achieved new state of the arts in almost every area of NLP, including character- level sequence to sequence transformation tasks like morphological inflection, especially when there are abundant labeled data (Goldberg, 2016). However, neural network models are usually very data-hungry, and the performance of such models can suffer when labeled data is limited. Unfortunately, the large amounts of labeled data needed is not always available and can be difficult to obtain for many languages. As interest has grown in low-resource NLP, several effective strategies to improve the performance of neural models have surfaced, and neural network models have become the dominant approach in low-resource settings as well. Such efforts for the morphological inflection task include engineering the neural network architecture to take better advantage of linguistic knowledge (Aharoni and Goldberg, 2017; Wu et al., 2018; Wu and Cotterell, 2019; Canby et al., 2020), designing data hallucination tech- niques to generate synthetic data based on existing labeled data (Silfverberg et al., 2017; Bergmanis et al., 2017; Anastasopoulos and Neubig, 2019; Yu et al., 2020), augmenting the training data by making better use of labeled or unlabeled data (Kann and Sch¨utze, …

Content truncated to 500 words