Neural Disambiguation of Lemma and Part of Speech in Morphologically Rich Languages

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 346–355 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 346 Using Multilingual Resources to Evaluate CEFRLex for Learner Applications Johannes Gra¨en1,2, David Alfter1, Gerold Schneider3,4 1Spr˚akbanken Text, Department of Swedish, University of Gothenburg 2Grael, Department of Translation and Language Sciences, Pompeu Fabra University 3Department of Computational Linguistics, University of Zurich 4English Department, University of Zurich {johannes.graen, david.alfter}@svenska.gu.se, gschneid@es.uzh.ch Abstract The Common European Framework of Reference for Languages (CEFR) defines six levels of learner proficiency, and links them to particular communicative abilities. The CEFRLex project aims at compiling lexical resources that link single words and multi-word expressions to particular CEFR levels. The resources are thought to reflect second language learner needs as they are compiled from CEFR-graded textbooks and other learner-directed texts. In this work, we investigate the applicability of CEFRLex resources for building language learning applications. Our main concerns were that vocabulary in language learning materials might be sparse, i.e. that not all vocabulary items that belong to a particular level would also occur in materials for that level, and, on the other hand, that vocabulary items might be used on lower-level materials if required by the topic (e.g. with a simpler paraphrasing or translation). Our results indicate that the English CEFRLex resource is in accordance with external resources that we jointly employ as gold standard. Together with other values obtained from monolingual and parallel corpora, we can indicate which entries need to be adjusted to obtain values that are even more in line with this gold standard. We expect that this finding also holds for the other languages. Keywords: Computer-Assisted Language Learning (CALL); Lexicon, Lexical Database; Multilinguality 1. Introduction Graded vocabulary lists have different applications such as serving as a basis for textbook writers, learner dictionaries or as self-paced learning tool for language learners (Kil- garriff et al., 2014). Especially in a second language learn- ing context, vocabulary knowledge is highly correlated with general language proficiency and is a prerequisite for suc- cessful communication (Nation, 2013). The main problem is that most graded vocabulary lists do not contain an evaluation of their quality and reliability. Nevertheless, as an user of such resources, one might want to know how reliable the resource is before employing it in the context of a language learning application. The Common European Framework of Reference (CEFR) for Languages (Council of Europe, 2001) is a scale of pro- ficiency divided into three broad levels, A, B, and C, each of which is further subdivided into two sub-levels, so that the full scale ranges over 6 levels, from A1 for beginning learners over A2, B1, B2, C1 to C2 for near-native learners. The most prominent use of the CEFR is in the form of (1) language certificates such as the Test of English as a Foreign Language (TOEFL) or International English Lan- guage Testing System (IELTS), and (2) CEFR-graded text- books. While most tests have their own scoring system, …

Content truncated to 500 words