Automatic Extraction of Learner Errors in ESL Sentences Using Linguistically Enhanced Alignments

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 825–835, Osaka, Japan, December 11-17 2016. Automatic Extraction of Learner Errors in ESL Sentences Using Linguistically Enhanced Alignments Mariano Felice Christopher Bryant Ted Briscoe ALTA Institute Computer Laboratory University of Cambridge Cambridge, UK {mf501, cjb255, ejb1}@cl.cam.ac.uk Abstract We propose a new method of automatically extracting learner errors from parallel English as a Second Language (ESL) sentences in an effort to regularise annotation formats and reduce inconsistencies. Specifically, given an original and corrected sentence, our method first uses a linguistically enhanced alignment algorithm to determine the most likely mappings between to- kens, and secondly employs a rule-based function to decide which alignments should be merged. Our method beats all previous approaches on the tested datasets, achieving state-of-the-art results for automatic error extraction. 1 Introduction Within the field of Machine Translation (MT), one of the first steps of data processing is to align a source sentence with a target sentence. This is necessary because we want to determine which tokens and phrases in the source language map to which equivalent tokens or phrases in the target language. As this would be extremely time consuming to do manually, several tools, such as GIZA++ (Och and Ney, 2003), have been made available to do this automatically. Within the related field of Grammatical Error Correction (GEC), we similarly want to align a source sentence with a target sentence to map errors to corrections (sometimes referred to as ‘correction detec- tion’; see example in Table 1). However, unlike in MT, the source and target sentences in GEC are in the same language and so a majority of tokens match. This means alignment is comparatively more straight- forward and so it is more feasible to annotate texts manually, rather than automatically. In fact two of the largest publicly available GEC datasets, the First Certificate in English (FCE) corpus (Yannakoudakis et al., 2011) and the National University of Singapore Corpus of Learner English (NUCLE) (Dahlmeier and Ng, 2012), were aligned and annotated manually. We took a guide tour on center city . We took a guided tour of the city center . Table 1: A sample alignment between an original uncorrected sentence and its corrected version. Nevertheless, automatic alignment of GEC data still has several advantages over manual alignment, not least because the latter is slow, laborious work. This is especially important for datasets that do not always contain explicit alignments, such as Lang-8 (Mizumoto et al., 2011), or GEC system output that needs to be aligned to the original uncorrected sentence. Another important benefit of an automatic alignment is that it tends to be more consistent than a human alignment. For example, within both the FCE and NUCLE, strings such as has eating are inconsistently corrected as [has → was] or [has eating → was eating] even though they fundamentally equate to the same thing. In fact, the latter seems less desirable given the token eating does not actually change. A similar case …

Content truncated to 500 words