Inflecting When There’s No Majority: Limitations of Encoder-Decoder Neural Networks as Cognitive Models for German Plurals

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1745–1756 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1745 Inflecting When There’s No Majority: Limitations of Encoder-Decoder Neural Networks as Cognitive Models for German Plurals Kate McCurdy Sharon Goldwater Adam Lopez Institute for Language, Cognition and Computation School of Informatics University of Edinburgh kate.mccurdy@ed.ac.uk, {sgwater, alopez}@inf.ed.ac.uk Abstract Can artificial neural networks learn to repre- sent inflectional morphology and generalize to new words as human speakers do? Kirov and Cotterell (2018) argue that the answer is yes: modern Encoder-Decoder (ED) architectures learn human-like behavior when inflecting En- glish verbs, such as extending the regular past tense form /-(e)d/ to novel words. However, their work does not address the criticism raised by Marcus et al. (1995): that neural models may learn to extend not the regular, but the most frequent class — and thus fail on tasks like German number inflection, where infre- quent suffixes like /-s/ can still be productively generalized. To investigate this question, we first collect a new dataset from German speak- ers (production and ratings of plural forms for novel nouns) that is designed to avoid sources of information unavailable to the ED model. The speaker data show high variability, and two suffixes evince ‘regular’ behavior, appear- ing more often with phonologically atypical in- puts. Encoder-decoder models do generalize the most frequently produced plural class, but do not show human-like variability or ‘regular’ extension of these other plural markers. We conclude that modern neural models may still struggle with minority-class generalization. 1 Introduction Morphology has historically been the site of vig- orous debate on the capacity of neural models to capture human speaker behavior, and hence ground claims about speaker cognition. In 1986, Rumel- hart and McClelland described a neural network model which learned to map English present tense verbs to their past tense forms. Importantly, the network handled both regular verbs, whose past tense is formed systematically by adding the suffix /-(e)d/ (e.g. jumped), and irregular verbs where the present and past tenses bear no systematic rela- tionship (e.g. ran). The authors suggested their model provided “an alternative [...] to the im- plicit knowledge of rules” (1986, 218), a claim which sparked considerable controversy. Pinker and Prince (1988) highlighted many empirical inad- equacies of the Rumelhart and McClelland model, and argued that these failures stemmed from “cen- tral features of connectionist ideology” and would persist in any neural network model lacking a sym- bolic processing component. Recently, however, Kirov and Cotterell (2018, henceforth K&C) revisited the English past tense debate and showed that modern recurrent neural networks with encoder-decoder (ED) architectures overcome many of the empirical limitations of ear- lier neural models. Their ED model successfully learns to generalize the regular past tense suffix /-(e)d/, achieving near-ceiling accuracy on held-out test data. Moreover, its errors result from over- application of the regular past tense (e.g. throw– throwed)—a type of error observed in human lan- guage learners as well—as opposed to the …

Content truncated to 500 words