Using Neural Machine Translation for Generating Diverse Challenging Exercises for Language Learner

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 6115–6129 July 9-14, 2023 ©2023 Association for Computational Linguistics Using Neural Machine Translation for Generating Diverse Challenging Exercises for Language Learners Frank Palma Gomez1 Subhadarshi Panda∗ Michael Flor2 Alla Rozovskaya1,3 1Queens College, CUNY 2Educational Testing Service 3CUNY Graduate Center frankpalma12@gmail.com, subhadarshipanda08@gmail.com mflor@ets.org, arozovskaya@qc.cuny.edu Abstract We propose a novel approach to automatically generate distractors for cloze exercises for En- glish language learners, using round-trip neu- ral machine translation. A carrier sentence is translated from English into another (pivot) language and back, and distractors are pro- duced by aligning the original sentence with its round-trip translation. We make use of 16 linguistically-diverse pivots and generate hun- dreds of translation hypotheses in each direc- tion. We show that using hundreds of transla- tions allows us to generate a rich set of challeng- ing distractors. Moreover, we find that typo- logically unrelated language pivots contribute more diverse candidate distractors, compared to language pivots that are closely related. We further evaluate the use of machine translation systems of varying quality and find that better quality MT systems produce more challenging distractors. Finally, we conduct a study with language learners, demonstrating that the auto- matically generated distractors are of the same difficulty as the gold distractors produced by human experts.1 1 Introduction A common challenge for language learners in- volves understanding how to appropriately use words that may have similar meanings but are used in different contexts. For instance, “main” and “vital” are semantically related but “main impor- tance” is not an acceptable expression while “vital importance” is. This subtle language knowledge is not explicitly available to learners. For this reason, word usage (collocation) errors are some of the most common types of errors even for advanced non-native speakers (Leacock et al., 2010). *Work was done while the author was at the CUNY Grad- uate Center. 1The code is available at https://github.com/ subhadarship/round-trip-distractors Carrier sentence Are these old plates of ______ importance or can I put them into storage? Target word: vital Valid distractors: main, urgent, lively Invalid distractors: great, utmost Table 1: A sentence for a fill-in-the-blank exercise with the target word “vital” removed. Multiple-choice list will include the target and 3 distractors. Examples of valid and invalid distractors are shown. In this work, we develop exercises for mastering vocabulary use for second (foreign) language learn- ers. We focus on cloze (fill-in-the-blank) exercises. A cloze exercise is a common method of teaching vocabulary, as well as assessing non-native speaker performance in a foreign language: a sentence is presented to the learner with one word (target) hid- den. The target word is presented along with a list of distractors (usually 3), and the learner should identify the target word from that list. Table 1 shows a sample cloze item with the target word “vital”. The carrier sentence along with a multiple- choice list is referred to as cloze item. A cloze (exercise) item is valid if exactly one word …

Content truncated to 500 words