Cross-lingual and Multilingual CLIP

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 697–706 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 697 LESPELL – A Multi-Lingual Benchmark Corpus of Spelling Errors to Develop Spellchecking Methods for Learner Language Marie Bexte, Ronja Laarmann-Quante, Andrea Horbach, Torsten Zesch Research Cluster “Digitalization, Diversity and Lifelong Learning – Consequences for Higher Education”, FernUniversit¨at in Hagen, Germany Abstract Spellchecking text written by language learners is especially challenging because errors made by learners differ both quanti- tatively and qualitatively from errors made by already proficient learners. We introduce LESPELL, a multi-lingual (English, German, Italian, and Czech) evaluation data set of spelling mistakes in context that we compiled from seven underlying learner corpora. Our experiments show that existing spellcheckers do not work well with learner data. Thus, we introduce a highly customizable spell checking component for the DKPro architecture, which improves performance in many settings. Keywords: spellchecking, error correction, language learner data, multilingual 1. Introduction Learner texts often differ both in the quantity and qual- ity of errors from texts written by native speakers (Rim- rott and Heift, 2008; Flor et al., 2015). Consider the following example, taken from the MERLIN-DE cor- pus (see Section 2): Ich schuche 3-4 zimm in Dreter Stock. The orthographically correct version would be Ich suche 3-4 Zimmer in dritter Stock. (‘I am look- ing for 3-4 rooms on third floor.’) Note that grammat- ical errors are not of interest here. We can see that all misspellings in this example have an edit distance greater than 1 to their corrections. The pronunciation, however, is often still very similar (e.g. Dreter [döe:t5] and dritter [döIt5]). Proficient writers, in contrast, of- ten produce typos which result in misspellings such as driktter, where an unrelated incorrect letter is intro- duced. Consequently, many downstream NLP tools are known to decrease in performance when applied on non- standard data such as learner language (Foster, 2010; Keiper et al., 2016). Off-the-shelf spellchecking tools are mostly targeted at errors of proficient writers and fail to correct multi-edit errors produced by learners (Rimrott and Heift, 2008). Therefore, there have been several attempts to build spellcheckers that are special- ized in learner errors, both for second language (L2, e.g. Boyd (2009)) and first language (L1) learners, i.e. young children (e.g. St¨uker et al. (2011), Downs et al. (2020)). The different approaches are hard to compare, however, because they have been evaluated on different data sets and partly on different languages. Therefore, it has not yet been determined to what extent the ap- proaches are transferable to other learner populations and other languages. In general, Flor et al. (2019) noted that most spellcheckers have been evaluated on either proprietary or artificial data. Therefore, they released TOEFL- Spell, a publicly available benchmark spelling data set which is based on real errors from L2 learners of En- glish. However, there is yet no comparable benchmark set for other languages or even for multiple languages sharing a common format. …

Content truncated to 500 words