Detecting Learner Errors in the Choice of Content Words Using Compositional Distributional Semantics

Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 1740–1751, Dublin, Ireland, August 23-29 2014. Detecting Learner Errors in the Choice of Content Words Using Compositional Distributional Semantics Ekaterina Kochmar Computer Laboratory University of Cambridge ek358@cl.cam.ac.uk Ted Briscoe Computer Laboratory University of Cambridge ejb@cl.cam.ac.uk Abstract We describe a novel approach to error detection in adjective–noun combinations. We present and release a new dataset of annotated errors where the examples are extracted from learner texts and annotated with error types. We show how compositional distributional semantic approaches can be applied to discriminate between correct and incorrect word combinations from learner data. Finally, we show how the output of the compositional distributional semantic models can be used as features in a classifier yielding good precision and accuracy. 1 Introduction The task of error detection and correction (henceforth, EDC) in non-native writing in English has been a focus of research in recent years. However, usually research in this area focuses on EDC in the use of function words, such as articles or prepositions (Leacock et al., 2010; Dale et al., 2012), while much less attention has been paid to errors in the choice of content words. Errors in function words are some of the most common error types in learner writing (Dalgish, 1985; Leacock et al., 2010), so it is important for any EDC system to be able to deal with such errors. Certain properties of these errors facilitate their detection and correction. As function words belong to closed classes, the set of possible corrections is limited by the size of the function word set. Since errors in function words are systematic and highly recurrent, in practice, each article or preposition has an even smaller number of appropriate alternatives. We illustrate this point with the following examples on (1) article and (2) preposition errors: (1) I am 0*/a student. (2) Last October, I came in*/to Tokyo. In (1) an EDC system would consider {a, an, the} as possible corrections for the missing article. To correct the preposition in in (2), an EDC system would consider the most frequent prepositions {on, from, for, of, about, to, at, with, by}, among which at or to would have a higher chance to be appropriate corrections as these are most often confused with in. Confusion sets can be learnt from learner texts, and probabilities can be set up according to the distribution of the confusions (Rozovskaya and Roth, 2011). EDC is usually cast as a multi-class classification task, with the number of classes equal to the number of target corrections. Detection and correction can occur simultaneously: an error is detected when an EDC system suggests using a word different from the one originally used by the learner, and the sug- gested word can be used as a correction. Each occurrence of a function word is represented with a feature vector, where features are derived from the surrounding context. This is usually highly informative for function words: for example, a context of I …

Content truncated to 500 words