Automatic Assessment of Student Translations for Foreign Language Tutoring

Proceedings of NAACL HLT 2007, pages 468–475, Rochester, NY, April 2007. c⃝2007 Association for Computational Linguistics Automatic Assessment of Student Translations for Foreign Language Tutoring Chao Wang and Stephanie Seneff Spoken Language Systems Group MIT Computer Science and Artificial Intelligence Laboratory The Stata Center, 32 Vassar Street, Cambridge, MA 02139 {wangc,seneff}@csail.mit.edu Abstract This paper introduces the use of speech translation technology for a new type of voice-interactive Computer Aided Lan- guage Learning (CALL) application. We describe a computer game we have devel- oped, in which the system presents sen- tences in a student’s native language to elicit spoken translations in the target new language. A critical technology is an al- gorithm to automatically verify the ap- propriateness of the student’s translation using linguistic analysis. Evaluation re- sults are presented on the system’s abil- ity to match human judgment of the cor- rectness of a student’s translation, for a set of 1115 utterances collected from 9 learn- ers of Mandarin Chinese translating flight domain sentences. We also demonstrate the effective use of context information to improve both recognition performance on non-native speech as well as the system’s accuracy in judging the translation quality. 1 Introduction It is widely recognized that one of the best ways to learn a foreign language is through spoken di- alogue with native speakers (Ehsani and Knodt, 1998). However, this is not a practical method in the classroom setting. A potential solution to this prob- lem is to rely on computer spoken dialogue systems to role play a tutor and/or a conversational partner. Ideally, a voice-interactive system can provide the learner with endless opportunities for practice and feedback. However, while a number of dialogue sys- tems have been developed (or adapted) for language learning purposes (Seneff et al., 2004; Johnson et al., 2004), the issues of speech understanding of the accented and disfluent utterances of a foreign lan- guage student typically lead to unacceptable perfor- mance (Eskenazi, 1999). A relatively successful application of speech pro- cessing technology is in the area of pronunciation training (Eskenazi, 1999; Witt, 1999; Hardison, 2004). In this case, a learner repeats words or sen- tences prompted by the computer, and receives feed- back on the segmental and suprasegmental quality of their speech. While such systems can be useful, they do not help the student learn how to formulate sentences within the language on their own. In this paper, we introduce a novel speech trans- lation game which aims to provide a fun and reli- able environment for language learners to gain pro- ficiency in communication. Our application is com- plementary to pronunciation training in that we pro- vide the opportunity for the students to practice ex- pressing themselves in the new language. While we don’t explicitly evaluate their pronunciations, we provide implicit feedback in that they must speak well enough for the speech recognizer to success- fully process their utterances. The translation exer- cise can also serve as a preparation stage in which students can practice relevant vocabulary and sen- tence constructs, …

Content truncated to 500 words