Generalising to German Plural Noun Classes, from the Perspective of a Recurrent Neural Network

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 94–108 November 10–11, 2021. ©2021 Association for Computational Linguistics 94 Generalising to German Plural Noun Classes, from the Perspective of a Recurrent Neural Network Verna Dankers⋆ , Anna Langedijk⋆ , Kate McCurdy , Adina Williams , Dieuwke Hupkes ILCC, University of Edinburgh ILLC, University of Amsterdam Facebook AI Research {vernadankers, annalangedijk}@gmail.com, kate.mccurdy@ed.ac.uk, {adinawilliams, dieuwkehupkes}@fb.com Abstract Inflectional morphology has since long been a useful testing ground for broader questions about generalisation in language and the via- bility of neural network models as cognitive models of language. Here, in line with that tradition, we explore how recurrent neural net- works acquire the complex German plural sys- tem and reflect upon how their strategy com- pares to human generalisation and rule-based models of this system. We perform analyses including behavioural experiments, diagnos- tic classification, representation analysis and causal interventions, suggesting that the mod- els rely on features that are also key predictors in rule-based models of German plurals. How- ever, the models also display shortcut learning, which is crucial to overcome in search of more cognitively plausible generalisation behaviour. 1 Introduction Language is a complex and mysterious system, which requires that speakers systematically gener- alise but also that they admit exceptions (Jackend- off and Audring, 2018; Bybee and Hopper, 2001; Pinker, 1998). A clear illustration of this is the do- main of morphology, where suffixes and affixes can be productively used to express a particular gram- matical property, but where there are also several words that follow irregular patterns for the same grammatical function. For example, while the past tense for most English verbs is formed by affixing -ed (walk → walked), the past tense of break is not breaked but broke. If an English speaker en- counters an unknown form such as treak, they must decide whether to attach -ed, or instead go with the irregular form troke. Precisely because of such intricacies, the computational task of acquiring a ⋆Equal contribution. <f> k a t z e </s> k a t z e n (-n) <f> m u t t i </s> <n> k l e i d </s> <m> w a g e n </s> w a g e n (-∅) k l e i d e r (-e r) m u t t i s (-s) <n> p f e r d </s> p f e r d e (-e) encoder decoder feminine neuter short length vowel Figure 1: Illustration of how our models predict plural nouns. Line thickness indicates performance per plu- ral class; colour gradients show how that performance increases while the encoder processes the word. morphological system capable of generalisation has a strong historical connection to cognitive science (e.g. Seidenberg and Plaut, 2014). Making progress on how human minds process this interesting subdomain of language, however, is a challenging enterprise: probing the internal repre- sentations of human minds is difficult and, in some cases, potentially unethical. Several researchers have therefore opted to instead investigate neural …

Content truncated to 500 words