A CALL System for Learning Preposition Usage

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 984–993, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics A CALL System for Learning Preposition Usage John Lee Department of Linguistics and Translation City University of Hong Kong jsylee@cityu.edu.hk Donald Sturgeon Fairbank Center for Chinese Studies Harvard University djs@dsturgeon.net Mengqi Luo Department of Linguistics and Translation City University of Hong Kong mengqluo@cityu.edu.hk Abstract Fill-in-the-blank items are commonly fea- tured in computer-assisted language learn- ing (CALL) systems. An item displays a sentence with a blank, and often proposes a number of choices for filling it. These choices should include one correct answer and several plausible distractors. We de- scribe a system that, given an English cor- pus, automatically generates distractors to produce items for preposition usage. We report a comprehensive evaluation on this system, involving both experts and learners. First, we analyze the diffi- culty levels of machine-generated carrier sentences and distractors, comparing sev- eral methods that exploit learner error and learner revision patterns. We show that the quality of machine-generated items ap- proaches that of human-crafted ones. Fur- ther, we investigate the extent to which mismatched L1 between the user and the learner corpora affects the quality of dis- tractors. Finally, we measure the system’s impact on the user’s language proficiency in both the short and the long term. 1 Introduction Fill-in-the-blank items, also known as gap-fill or cloze items, are a common form of exercise in computer-assisted language learning (CALL) ap- plications. Table 1 shows an example item de- signed for teaching English preposition usage. It contains a sentence, “The objective is to kick the ball into the opponent’s goal”, with the preposi- tion “into” blanked out; this sentence serves as the stem (or carrier sentence). It is followed by four choices for the blank, one of which is the key (i.e., the correct answer), and the other three are dis- tractors. These choices enable the CALL applica- tion to provide immediate and objective feedback to the learner. A high-quality item must meet multiple re- quirements. It should have a stem that is fluent and matches the reading ability of the learner; a blank that is appropriate for the intended peda- gogical goal; exactly one correct answer among the choices offered; and finally, a number of dis- tractors that seem plausible to the learner, and yet would each yield an incorrect sentence. Relying on language teachers to author these items is time consuming. Automatic generation of these items would not only expedite item authoring, but also potentially provide personalized items to suit the needs of individual learners. This paper addresses two research topics: • How do machine-generated items compare with human-crafted items in terms of their quality? • Do these items help improve the users’ lan- guage proficiency? For the first question, we focus on automatic generation of preposition distractors, comparing three different methods for distractor generation. One is based on word co-occurrence in standard The objective is to kick the ball the opponent’s …

Content truncated to 500 words