Evaluation of the KIT Lecture Translation System

Evaluation of the KIT Lecture Translation System Markus M¨uller⋆, Sarah F¨unfer†, Sebastian St¨uker⋆, Alex Waibel⋆ ⋆Karlsruhe Institute of Technology, Adenauerring 2, 76131 Karlsruhe, Germany †Freelancer, Essenweinstr. 33, 76131 Karlsruhe, Germany m.mueller@kit.edu, europeersarah@gmx.de, sebastian.stueker@kit.edu, waibel@kit.edu Abstract To attract many foreign students is among the goals of the Karlsruhe Institute of Technology (KIT). One obstacle to achieving this goal is the fact that lectures at KIT are usually held in German which many foreign students are not sufficiently proficient in, as, e.g., opposed to English. While the students from abroad are learning German during their stay at KIT, it is very challenging to become proficient enough in it in order to follow such a complex communication situation as a lecture. As a solution to this problem we offer our automatic simultaneous lecture translation at KIT’s lecture halls which automatically translates the German lectures into English in real time for the students. While not as good as human interpreters, the system is available at a price that KIT can afford in order to offer it in potentially all lectures. In order to assess whether the quality of the system is high enough in order to be of use to our foreign students at KIT we have conducted a user study on the benefit of the system to its users over the course of two terms. In this paper we present this study, the way it was conducted and its results. As it turns out the results indicate that the quality of the system has passed a threshold as to be able to support students in their studies. The detailed feedback participants to the study have helped to identify the most crucial weaknesses of the systems and has guided which development steps to take next. Keywords: automatic speech recognition, machine translation, user study 1. Introduction Many students studying at KIT are from abroad. While they usually speak English fluently, they are often not suf- ficiently proficient in German in order to follow the content of German lectures. Since most lectures at KIT are given in German, this means that students from abroad struggle with the language barrier during their stay at KIT, most promi- nently in their academic studies. Universities in English speaking countries do not suffer from this problem, hence they have a higher percentage of foreign students. One of the goals of KIT’s internationalization strategy is to become more attractive for students from abroad, among others by lowering the language barrier while at the same time still teaching in German due to considerations of cultural and scientific diversity. In this the use of human interpreters for translating lectures at KIT is not a feasible option, as unlike the European Par- liament, KIT does not have the financial resources for em- ploying sufficient amounts of human interpreters. Instead, we have started to offer an automatic simultaneous trans- lation system at several of KIT’s lecture halls (Cho et al., 2013). The system translates the German lectures into En- glish in real time and displays …

Content truncated to 500 words