An SLA Corpus Annotated with Pedagogically Relevant Grammatical Structures

An SLA Corpus Annotated with Pedagogically Relevant Grammatical Structures Leonardo Zilio, Rodrigo Wilkens and C´edrick Fairon Centre de Traitement Automatique du Langage - CENTAL Universit´e catholique de Louvain - UCL {leonardo.zilio, rodrigo.wilkens, cedrick.fairon}@uclouvain.be Abstract The evaluation of a language learner’s proficiency in second language is a task that normally involves comparing the learner’s production with a learning framework of the target language. One of the most well known frameworks is the Common European Framework for Languages (CEFR), which addresses language learning in general and is broadly used in the European Union, while serving as reference in countries outside the EU as well. In this study, we automatically annotated a corpus of texts produced by language learners with pedagogically relevant grammatical structures and observed how these structures are being employed by learners from different proficiency levels. We analyzed the use of structures both in terms of evolution along the levels and in terms of level in which the structures are used the most. The annotated resource, SGATe, presents a rich source of information for teachers that wish to compare the production of their students with those of already certified language learners. Keywords: SLA, syntactic annotation, CEFR, EFCAMDAT, learner profile 1. Introduction The evaluation of a language learner’s capacity when pro- ducing texts in a foreign language is not an easy task. The factors that impact the overall categorization of the pro- duced text are many, roughly ranging from vocabulary to discourse strategies, passing by syntax and semantics. One way of facilitating this task is to have a profile of the lan- guage learner skills, so that there are some hints on what to expect from a learner in each language level. For this reason, there are different frameworks that orga- nize the order in which the different language skills should be targeted at each step, while also indicating the required skills for the evaluation of a learner’s proficiency. Exam- ples of this type of frameworks are the Common Euro- pean Framework for Languages (CEFR) and the Cambridge ESOL, which are based on levels, and IELTS and TOEFL, which are based on scores. These frameworks pinpoint, in differently organized fashion, how it is expected that the second language learning will take place for the learner, by listing skills and associating them with an expected level (or score). In this study, we use a different approach. Instead of point- ing out the skills that the learner should be able to master in order to be evaluated as having achieved a certain level of proficiency, our objective is to look directly at the pro- duction of learners that have already been evaluated as hav- ing achieved a certain degree of proficiency. By investigat- ing texts produced by learners and quantitatively observing how different types of grammatical structures are used by learners from different language levels and by analyzing the distribution of grammatical structures in their textual pro- duction, we aim at finding out which structures are more or less active and how they evolve …

Content truncated to 500 words