A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1933 - 1945 December 7-11, 2022 ©2022 Association for Computational Linguistics A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflection Adam Wiemerslage and Shiran Dudy and Katharina Kann University of Colorado Boulder first.last@colorado.edu Abstract Neural networks have long been at the center of a debate around the cognitive mechanism by which humans process inflectional morphol- ogy. This debate has gravitated into NLP by way of the question: Are neural networks a fea- sible account for human behavior in morpho- logical inflection? We address that question by measuring the correlation between human judgments and neural network probabilities for unknown word inflections. We test a larger range of architectures than previously studied on two important tasks for the cognitive pro- cessing debate: English past tense, and Ger- man number inflection. We find evidence that the Transformer may be a better account of hu- man behavior than LSTMs on these datasets, and that LSTM features known to increase in- flection accuracy do not always result in more human-like behavior. 1 Introduction: The Past Tense Debate Morphological inflection has historically been a proving ground for studying models of language acquisition. Rumelhart and McClelland (1985) famously presented a neural network that they claimed could learn English past tense inflection. However, Pinker and Prince (1988) proposed a dual-route theory for inflection, wherein regular verbs are inflected based on rules and irregular verbs are looked up in the lexicon. They high- lighted several shortcomings of Rumelhart and Mc- Clelland (1985) that they claimed any neural net- work would suffer from. This opened a line of work wherein cognitive theories of inflection are analyzed by implement- ing them as computational models and comparing their behavior to that of humans. A famous study in the area of morphology is the wug test (Berko, 1958), where human participants are prompted with a novel-to-them nonce word and asked to produce its plural form. Similarly, morphological inflection models are generally evaluated on words they have R&M M95 P&P K&C A&H Corkery et al Evidence for Evidence against Ours Dankers et al McCurdy et al Figure 1: Summary of the past tense debate as it per- tains to this work, color coded by evidence for (blue) or against (red) neural networks as a cognitively plausi- ble account for human behavior. not seen during training. However, since they are evaluated on actual words, it is impossible to mean- ingfully ask a native speaker, who knows the words’ inflected forms, how likely different reasonable in- flections for the words in a model’s test data are. Thus, in order to compare the behavior of humans and models on words unknown to both, prior work has created sets of made-up nonce words (Marcus et al., 1995; Albright and Hayes, 2003). English Past Tense English verbs inflect to ex- press the past and present tense distinction. Most verbs inflect for past tense by applying the /-d/, /-Id/, or /-t/ suffix: allophones of the …

Content truncated to 500 words