Improving POS Tagging of German Learner Language in a Reading Comprehension Scenario

Improving POS Tagging of German Learner Language in a Reading Comprehension Scenario Lena Keiper, Andrea Horbach, Stefan Thater Saarland University Department of Computational Linguistics Saarbr¨ucken, Germany {lkeiper, andrea, stth}@coli.uni-saarland.de Abstract We present a novel method to automatically improve the accurrcy of part-of-speech taggers on learner language. The key idea underlying our approach is to exploit the structure of a typical language learner task and automatically induce POS information for out-of-vocabulary (OOV) words. To evaluate the effectiveness of our approach, we add manual POS and normalization information to an existing language learner corpus. Our evaluation shows an increase in accuracy from 72.4% to 81.5% on OOV words. Keywords: Part-of-Speech Tagging, Learner Language, Corpus Annotation 1. Introduction With the increasing availability of computer-assisted lan- guage learning (CALL) applications, there comes a growing need to develop and improve NLP tools that are able to process learner language automatically in order to give meaningful feedback to language learners. For most tasks involving free-text input by the learner (unlike for instance simple multiple choice questions or gap-filling tasks), appli- cations need to deal with unrestricted learner language input using NLP techniques. In this paper, we consider the preprocessing step of part-of- speech (POS) tagging of learner language. POS tagging is a fundamental part of most NLP tool chains and provides necessary input for higher-level processing steps such as algorithms for scoring the contents of learner answers. The challenge is that out-of-the-box POS tagging models are usually trained on standard language like newspaper arti- cles, and consequently also perform best on newspaper text. Learner language, however, typically differs substantially from the newspaper data, so that out-of-the-box-models are not directly applicable in a CALL setting. The standard model of the tagger considered in this paper, for instance, achieves an accuracy of over 97% when applied to newspa- per text, but only 93% when applied to learner language. One important reason for this performance drop is that learner language contains many out-of-vocabulary (OOV) tokens, i.e. tokens that the tagger has not seen during train- ing. They are frequent in learner language for two reasons: One is that the domain of the learner language input differs from typical newswire texts. We call these OOV words lex- ical gaps. The second and more important reason is that learner language contains a lot of noise such as typos and other spelling errors, as well as grammar problems. They lead to nonexistent word forms which we call misspellings. Since learner language also tends to differ on the syntactic level, the tagger often cannot exploit contextual information to guess the correct POS tag for OOV as effectively as in the case of standard language. Examples (1) and (2) illustrate these phenomena: The first example shows two typical types of misspelling errors at the lexical level: The noun “Erflog” (correct form: Erfolg, Eng.: success) is a spelling error with a letter swap; “geld” (correct form: Geld, Eng.: money) is written in lowercase although in German all nouns are capitalized. Due to these errors a …

Content truncated to 500 words