RNN Simulations of Grammaticality Judgments on Long-distance Dependencies

Proceedings of the 27th International Conference on Computational Linguistics, pages 133–144 Santa Fe, New Mexico, USA, August 20-26, 2018. 133 RNN Simulations of Grammaticality Judgments on Long-distance Dependencies Shammur Absar Chowdhury and Roberto Zamparelli CIMeC: Center for Mind/Brain Sciences University of Trento {shammur.chowdhury, roberto.zamparelli}@unitn.it Abstract The paper explores the ability of LSTM networks trained on a language modeling task to detect linguistic structures which are ungrammatical due to extraction violations (extra argu- ments and subject-relative clause island violations), and considers its implications for the de- bate on language innatism. The results show that the current RNN model can correctly classify (un)grammatical sentences, in certain conditions, but it is sensitive to linguistic processing fac- tors and probably ultimately unable to induce a more abstract notion of grammaticality, at least in the domain we tested. Title and Abstract in Italian RNN Simulazioni di giudizi di grammaticit`a sulle dipendenze a distanza L’articolo studia la capacit`a delle reti neurali LSTM addestrate su un compito di modellazione linguistica di rilevare strutture linguistiche che sono agrammaticali a causa di violazioni nella estrazione di argomenti (dovute alla presenza di argomenti di troppo, o alla presenza di isole del soggetto e delle frasi relative), esplorando le implicazioni per il dibattito sull’innatismo linguis- tico. I risultati mostrano che l’attuale modello RNN pu`o classificare correttamente frasi gram- maticali, in certe condizioni, ma `e eccessivamente sensibile a fattori di elaborazione linguistica e probabilmente non in grado di indurre una nozione pi`u astratta di grammaticalit`a, almeno nel dominio da noi testato. 1 Introduction Native speaker intuitions about the meaning and grammaticality of linguistic expressions have been the key methodology in theoretical linguistics since at least Chomsky (1957), and are widely seen as a crucial window on the internalized linguistic competence which theoretical linguistics aims to study. Despite lively discussions on the limits of the methodology (see Cowart, 1997; Sprouse and Almeida, 2012; Sprouse et al., 2013, 2016), the availability of very large corpora has not replaced in linguistics the need for judgments on artificially constructed cases whenever theoretical points hinge on the status of very rare or complex constructions, concern languages or dialects for which large corpora do not exist, or involve semantic intuitions that have no easily detectable correlates in corpora (e.g. semantic scope alternations). Due to its scarce practical applications, modeling grammaticality judgments with computers has never been a typical NLP task, but it can be an important testbed for theories of language processing and grammatical competence. In particular, judgments on syntactic well-formedness require a sensitivity to long-distance structural cues which is a crucial aspect of language competence (Everaert et al., 2015). Elman’s (1991) pioneering work on the application of simple recurrent neural network (RNN, Elman 1990) to linguistic sequences showed that such networks could indeed learn some linguistic structures, but had a tendency to forget important linguistic features (e.g. the presence of a Wh-element) as new This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ 134 material arrived. However, more recent network …

Content truncated to 500 words