Korean L2 Vocabulary Prediction: Can a Large Annotated Corpus be Used to Train Better Models for Predicting Unknown Words?

Korean L2 Vocabulary Prediction: Can a Large Annotated Corpus be Used to Train Better Models for Predicting Unknown Words? Kevin P. Yancey, Yves Lepage Waseda University 2-7 Hibikino, Wakamatu-ku, Kitakyushu city, Fukuoka, Japan kpyancey@fuji.waseda.jp, yves.lepage@waseda.jp Abstract Vocabulary knowledge prediction is an important task in lexical text simplification for foreign language learners (L2 learners). However, previously studied methods that use hand-crafted rules based on one or two word features have had limited success. A recent study hypothesized that a supervised learning classifier trained on a large annotated corpus of words unknown by L2 learners may yield better results. Our study crowdsourced the production of such a corpus for Korean, now consisting of 2,385 annotated passages contributed by 357 distinct L2 learners. Our preliminary evaluation of models trained on this corpus show favorable results, thus confirming the hypothesis. In this paper, we describe our methodology for building this resource in detail and analyze its results so that it can be duplicated for other languages. We also present our preliminary evaluation of models trained on this annotated corpus, the best of which recalls 80 % of unknown words with 71 % precision. We make our annotation data available. Keywords: lexical simplification, vocabulary knowledge prediction, crowdsourcing 1. Introduction Our goal is to build a text simplification system for L2 learners of Korean to aid reading comprehension and ex- pedite language acquisition through reading. Reading, and extensive reading in particular (i.e., the practice of reading large amounts of easy, entertaining text), has been shown to have many benefits to language acquisition, including improving grammar (Krashen, 2003) , writing (Mason and Krashen, 1997), and listening skills (Elley, 1991), as well as reading proficiency and motivation (Crawford Camiciot- toli, 2001). A task critical to simplifying texts for L2 learners is esti- mating the proficiency level required to understand a word (i.e., the word’s “complexity”), so that unknown words can be predicted and simpler replacements can be selected. Many past proposals focused on rules that used word fre- quency, length, or some combination thereof as a proxy for word complexity (Devlin and Tait, 1998; Bott et al., 2012; Shardlow, 2013), but these features do not correlate per- fectly with word complexity for L2 learners. More recently, Tack et al. (2016) proposed a model that measured a word’s complexity as its first level of occurrence within a corpus of graded textbooks for L2 learners, evaluating it against a corpus annotated by four L2 learners of French. However, their model was only able to correctly classify 40 % of the unknown words. Tack et al.’s paper hypothesized that better results might be obtained by compiling a much larger annotated corpus and casting the problem as a supervised learning problem. In this paper, we will explore this alternative. We will de- scribe our methodology for constructing a large annotated corpus by crowdsourcing via the Internet, and will evalu- ate the predictive capability of several models built from this corpus via supervised learning, comparing our results to previous approaches. Finally, we will …

Content truncated to 500 words