Czech Grammar Error Correction with a Large and Diverse Corpus

Czech Grammar Error Correction with a Large and Diverse Corpus Jakub N´aplava† Milan Straka† Jana Strakov´a† Alexandr Rosen‡ †Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics, Czech Republic {naplava,straka,strakova}@ufal.mff.cuni.cz ‡Charles University, Faculty of Arts Institute of Theoretical and Computational Linguistics, Czech Republic alexandr.rosen@ff.cuni.cz Abstract We introduce a large and diverse Czech cor- pus annotated for grammatical error correction (GEC) with the aim to contribute to the still scarce data resources in this domain for lan- guages other than English. The Grammar Error Correction Corpus for Czech (GECCC) offers a variety of four domains, covering error distributions ranging from high error density essays written by non-native speakers, to web- site texts, where errors are expected to be much less common. We compare several Czech GEC systems, including several Transformer-based ones, setting a strong baseline to future re- search. Finally, we meta-evaluate common GEC metrics against human judgments on our data. We make the new Czech GEC corpus publicly available under the CC BY-SA 4.0 li- cense at http://hdl.handle.net/11234 /1-4639. 1 Introduction Representative data both in terms of size and domain coverage are vital for NLP systems devel- opment. However, in the field of grammar error correction (GEC), most GEC corpora are limited to corrections of mistakes made by foreign or second language learners even in the case of En- glish (Tajiri et al., 2012; Dahlmeier et al., 2013; Yannakoudakis et al., 2011, 2018; Ng et al., 2014; Napoles et al., 2017). At the same time, as recently pointed out by Flachs et al. (2020), learner cor- pora are only a part of the full spectrum of GEC applications. To alleviate the skewed perspective, the authors released a corpus of website texts. Despite recent efforts aimed to mitigate the notorious shortage of national GEC-annotated cor- pora (Boyd, 2018; Rozovskaya and Roth, 2019; Davidson et al., 2020; Syvokon and Nahorna, 2021; Cotet et al., 2020; N´aplava and Straka, 2019), the lack of adequate data is even more acute in languages other than English. We aim to address both the issue of scarcity of non-English data and the ubiquitous need for broad domain coverage by presenting a new, large and diverse Czech corpus, expertly annotated for GEC. Grammar Error Correction Corpus for Czech (GECCC) includes texts from multiple domains in a total of 83 058 sentences, being, to our knowl- edge, the largest non-English GEC corpus, as well as being one of the largest GEC corpora overall. In order to represent a diversity of writing styles and origins, besides essays of both native and non-native speakers from Czech learner cor- pora, we also scraped website texts to complement the learner domain with supposedly lower error density texts, encompassing a representation of the following four domains: • Native Formal – essays written by native students of elementary and secondary schools • Native Web Informal – informal website discussions • Romani – essays written by children and teenagers of the Romani ethnic minority • Second Learners – essays written by non- …

Content truncated to 500 words