‘Calling on the classical phone’: a distributional model of adjective-noun errors in learners’ English

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 976–986, Osaka, Japan, December 11-17 2016. ‘Calling on the classical phone’: a distributional model of adjective-noun errors in learners’ English Aur´elie Herbelot Centre for Mind/Brain Sciences University of Trento aurelie.herbelot@unitn.it Ekaterina Kochmar ALTA Institute University of Cambridge ek358@cl.cam.ac.uk Abstract In this paper we discuss three key points related to error detection (ED) in learners’ English. We focus on content word ED as one of the most challenging tasks in this area, illustrating our claims on adjective–noun (AN) combinations. In particular, we (1) investigate the role of con- text in accurately capturing semantic anomalies and implement a system based on distributional topic coherence, which achieves state-of-the-art accuracy on a standard test set; (2) thoroughly investigate our system’s performance across individual adjective classes, concluding that a class- dependent approach is beneficial to the task; (3) discuss the data size bottleneck in this area, and highlight the challenges of automatic error generation for content words. 1 Introduction Error detection (ED) in the prose of ‘English as a Second Language’ (ESL) learners has recently attracted much attention (Ng et al., 2014; Ng et al., 2013; Dale et al., 2012; Dale and Kilgarriff, 2011). Earlier work on ED in ESL writing mostly focused on grammatical errors and errors in function words (Felice and Pulman, 2008; Gamon et al., 2008; Tetreault et al., 2010; Gamon, 2010; Rozovskaya and Roth, 2010a; Dahlmeier and Ng, 2011b; Ng et al., 2013). Lately, the focus has shifted to other error types, with the recent shared tasks encompassing all errors (Ng et al., 2014; Daudaravicius et al., 2016). In Ng et al. (2014), errors in content words are reported to be the second most frequent error type among 28 categories, accounting for 11.8% of all errors in the training and for about 14% in the test data, yet most teams scored poorly in this category suggesting that this is a challenging and mostly unsolved problem. Current ED approaches can be broadly described as either modular, addressing one error type in par- ticular, or as comprehensive, spanning all error types, as in case of the SMT-based techniques (Felice et al., 2014; Junczys-Dowmunt and Grundkiewicz, 2014). The modular approaches rely on the systematic and recurrent nature of the error patterns, and on the availability of closed confusion sets which en- able casting the task as a multi-class classification problem. Since content words do not assume a finite set of confusions, it has been shown that ED for these combinations cannot be performed in a similar way (Kochmar and Briscoe, 2014; Rozovskaya et al., 2014). State-of-the-art SMT-based approaches also struggle with content word errors. We argue that ED systems for words which carry lexical meaning should necessarily involve a semantic component, which is typically not needed for other error types. From a pedagogical point of view, detecting content word errors is an important task. Since content words carry the semantics of a sentence as well as the communicative intent of …

Content truncated to 500 words