Privacy by Design and Language Resources

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 332–339 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 332 A Dataset for Investigating the Impact of Feedback on Student Revision Outcome Ildik´o Pil´an1*, John Lee2, Chak Yan Yeung2, Jonathan Webster2 1Department of Informatics, University of Oslo 2Department of Linguistics and Translation, City University of Hong Kong ildikop@ifi.uio.no, jsylee@cityu.edu.hk, cyyeung91@gmail.com, ctjjw@cityu.edu.hk Abstract We present an annotation scheme and a dataset of teacher feedback provided for texts written by non-native speakers of English. The dataset consists of student-written sentences in their original and revised versions with teacher feedback provided for the errors. Feedback appears both in the form of open-ended comments and error category tags. We focus on a specific error type, namely linking adverbial (e.g. however, moreover) errors. The dataset has been annotated for two aspects: (i) revision outcome establishing whether the re-written student sentence was correct and (ii) directness, indicating whether teachers provided explicitly the correction in their feedback. This dataset allows for studies around the characteristics of teacher feedback and how these influence students’ revision outcome. We describe the data preparation process and we present initial statistical investigations regarding the effect of different feedback characteristics on revision outcome. These show that open-ended comments and mitigating expressions appear in a higher proportion of successful revisions than unsuccessful ones, while directness and metalinguistic terms have no effect. Given that the use of this type of data is relatively unexplored in natural language processing (NLP) applications, we also report some observations and challenges when working with feedback data. Keywords: corrective feedback, hedging, error correction, Intelligent Computer-Assisted Language Learning 1. Introduction Feedback has an important role in facilitating the learning process (Gass and Mackey, 2013). Information about the occurrence of an error, explanation about why it occurred and suggestions about how to correct it can help language learners revise incorrectly used linguistic elements and it can contribute to avoiding repeated future occurrences of these errors. Despite numerous investigations on the kinds of feedback that can best promote learning, this question remains an open debate in the area of Second Language Acquisition (SLA) (Lightbown and Spada, 1990; Lyster and Ranta, 1997; Sheen and Ellis, 2011). The application of computational linguistic methods to answer this ques- tion remains much less explored (Nagata, 2019). Since these methods allow for the automatic processing of larger amounts of texts, it has a good potential to contribute to this debate with empirical evidence. To promote research on feedback in the above mentioned areas, we collect and annotate a set of teacher feedback and pairs of original and revised student-written sentences from a corpus of teacher-corrected academic essays authored by non-native speakers of English1. We annotate two types of information: (i) feedback directness, whether the feed- back explicitly contained the correction (direct) or not (in- direct), and (ii) revision outcome, i.e. whether an error was successfully revised by a student or not. The resource en- ables investigations about which characteristics of …

Content truncated to 500 words