Classifying Syntactic Errors in Learner Language

Proceedings of the 24th Conference on Computational Natural Language Learning, pages 97–107 Online, November 19-20, 2020. c⃝2020 Association for Computational Linguistics https://doi.org/10.18653/v1/P17 97 Classifying Syntactic Errors in Learner Language Leshem Choshen∗ Department of Computer Science Hebrew University of Jerusalem leshem.choshen@mail.huji.ac.il Dmitry Nikolaev∗ Department of Linguistics Stockholm University dnikolaev@fastmail.com Yevgeni Berzak BCS MIT berzak@mit.edu Omri Abend Department of Computer Science Hebrew University of Jerusalem omri.abend@mail.huji.ac.il Abstract We present a method for classifying syntac- tic errors in learner language, namely errors whose correction alters the morphosyntactic structure of a sentence. The methodology builds on the established Universal Depen- dencies syntactic representation scheme, and provides complementary information to other error-classification systems. Unlike existing error classification methods, our method is applicable across languages, which we show- case by producing a detailed picture of syntac- tic errors in learner English and learner Rus- sian. We further demonstrate the utility of the methodology for analyzing the outputs of lead- ing Grammatical Error Correction (GEC) sys- tems. 1 Introduction Taxonomies of grammatical errors are important for linguistic and computational analysis of learner language, as well as for Grammatical Error Cor- rection (GEC) systems.1 Such taxonomies divide the complex space of errors into meaningful cate- gories and enable characterizing their distribution in learner productions. This information can be beneficial for GEC: it can support the development of systems that focus on specific error types, serve as a form of inductive bias (for example, by regu- larizing the system’s output to have a desired dis- tribution over correction types), and guide data augmentation and data filtering by controlling the distribution of error types. Error taxonomies can also improve the interpretability of system outputs for error analysis and learner feedback. A number of annotation efforts for learner lan- guage developed error taxonomies (Nicholls, 2003; Dahlmeier et al., 2013), and statistical classifiers ∗First two authors contributed equally. 1Code can be found in github repo GEC UD divergences. Matrices directly mentioned are included in the appendix. ... if you like a subject you ’ll study it easierADJ xcomp Source: ... if you like a subject you ’ll study it more easilyADV advmod advmod Reference: Figure 1: Example of an edit of type ADJ � ADV in POS terms and xcomp � advmod in edge-label terms. Corre- sponding spans are boldfaced. into such taxonomies, notably ERRANT (Bryant et al., 2017). Taking error types into consideration in learning has also been shown to improve GEC performance (Kantor et al., 2019, cf. §6). However, most existing taxonomies are fairly coarse-grained and language specific, and do not produce mean- ingful types for a large proportion of the errors. For example, 25% of the errors in the standard NUCLE corpus (Dahlmeier et al., 2013) are mapped to the residual category OTHER (see §3.3). We propose SERCL, a taxonomy of Syntactic Errors (SEs) and an automatic Classification. In- spired by a longstanding tradition in Machine Translation (MT) which analyses divergences be- tween source and translated texts based on syntac- tic structure (Dorr, 1994; Nikolaev et al., 2020), SERCL …

Content truncated to 500 words