Improving Machine Translation of Educational Content via Crowdsourcing

Improving Machine Translation of Educational Content via Crowdsourcing Maximiliana Behnke1, Antonio Valerio Miceli Barone1, Rico Sennrich1, Vilelmini Sosoni2, Thanasis Naskos3, Eirini Takoulidou3, Maria Stasimioti2, Menno van Zaanen4, Sheila Castilho5 Federico Gaspari5, Panayota Georgakopoulou6 Valia Kordoni7, Markus Egg7, Katia Lida Kermanidis3 1School of Informatics, The University of Edinburgh 2Department of Foreign Languages, Translation and Interpreting, Ionian University 3Department of Informatics, Ionian University 4Department of Communication and Information Sciences,Tilburg University 5ADAPT Centre, Dublin City University 6Deluxe Media Europe 7Humboldt-Universität zu Berlin, Germany maximiliana.behnke@ed.ac.uk; amiceli@inf.ed.ac.uk; rico.sennrich@ed.ac.uk; vilelmini@hotmail.com; anaskos@ionio.gr; rinoulit@gmail.com; stasimioti.maria@gmail.com; mvzaanen@uvt.nl; sheila.castilho@adaptcentre.ie; federico.gaspari@adaptcentre.ie; yota.georgakopoulou@bydeluxe.com; kordonie@anglistik.hu-berlin.de; markus.egg@anglistik.hu-berlin.de; kerman@ionio.gr Abstract The limited availability of in-domain training data is a major issue in the training of application-specific neural machine translation models. Professional outsourcing of bilingual data collections is costly and often not feasible. In this paper we analyze the influence of using crowdsourcing as a scalable way to obtain translations of target in-domain data having in mind that the translations can be of a lower quality. We apply crowdsourcing with carefully designed quality controls to create parallel corpora for the educational domain by collecting translations of texts from MOOCs from English to eleven languages, which we then use to fine-tune neural machine translation models previously trained on general-domain data. The results from our research indicate that crowdsourced data collected with proper quality controls consistently yields performance gains over general-domain baseline systems, and systems fine-tuned with pre-existing in-domain corpora. Keywords: MOOCs, neural machine translation, crowdsourcing 1. Introduction The European Union Horizon 2020 TraMOOC project (Translation for Massive Open Online Courses) aims at enhancing multilingual access to online education by providing machine translation solutions for the educational content available in MOOCs, i.e. video lecture subtitles, slides, assignments, quiz text, and course discussion forum text (Kordoni et al., 2016). Educational content is translated from English into eleven European and BRIC languages (Bulgarian, Chinese, Croatian, Czech, Dutch, German, Greek, Italian, Polish, Portuguese, and Russian). This specific domain, i.e. the content of online courses, imposes a set of challenging properties, such as extensive use of domain-specific terms and entities, frequent occurrence of unknown words, subtitle segmentation, spoken language characteristics and social web text properties. Achieving high-quality machine translation in these conditions therefore requires significant amounts of in-domain data for training and testing. Creating such data by hiring professional translators would be expensive, especially considering that it would have to be done for eleven target languages. Therefore we turn our attention to crowdsourcing as a cost-saving alternative. The impact of non-expert input, i.e. translations provided by non-professional translators, on the development and the evaluation of machine translation engines has been investigated in previous research (Callison-Burch, 2009; Zaidan and Callison-Burch, 2011; Ambati, 2012). Crowdsourcing has been used to this end, and the main research concern has been whether input by the general crowd, that has no expertise in linguistics or translation studies, can improve the quality of large-scale machine translation systems in a manner that is cost- and time- effective. In this work we use crowdsourcing with carefully designed quality controls to collect …

Content truncated to 500 words