Multi-Task Modeling of Phonographic Languages: Translating Middle Egyptian Hieroglyphs

The IWSLT 2019 KIT Speech Translation System †Ngoc-Quan Pham,†Thai-Son Nguyen, †Thanh-Le Ha, †Juan Hussain, †Felix Schneider, ‡Jan Niehues, †Sebastian St¨uker, ∗†Alexander Waibel †Institute for Anthropomatics and Robotics KIT - Karlsruhe Institute of Technology, Germany firstname.lastname@kit.edu ‡Department of Data Science University of Maastricht, Netherlands firstname.lastname@maastrichtuniversity.nl ∗ Language Technologies Institute Carnegie Mellon University firstname.lastname@cmu.edu Abstract This paper describes KIT’s submission to the IWSLT 2019 Speech Translation task on two sub-tasks corresponding to two different datasets. We investigate different end-to-end architectures for the speech recognition module, including our new transformer-based architectures. Overall, our mod- ules in the pipe-line are based on the transformer architec- ture which has recently achieved great results in various fields. In our systems, using transformer is also advantageous compared to traditional hybrid systems in term of simplicity while still having competent results. 1. Introduction The Karlsruhe Institute of Technology (KIT) participated in the IWSLT 2019 Evaluation Campaign in two tasks: Speech Translation task (SLT) and Text Translation. This paper re- ports KIT systems for SLT task. 1.1. SLT Task Different from previous years, this year’s IWSLT SLT task focuses on the end-to-end performance of the speech transla- tion systems on two datasets, or as we called them two sub- tasks: How2[1] and TED. This makes ways for building end- to-end systems as well as multi-modal systems since the par- ticipants can use different modalities such as speech, text and even video in training the translation systems. For the TED dataset, speech, alignment information and text data are pro- vided within the MuST-C corpus[2] in English-German. For the How2 dataset, which is built based on the instructional videos and their transcripts in English and Portuguese, the visual-grounded information is also provided. We have still, however, conducted our SLT systems for both datasets in a piper-lined manner: The input would go through our Speech Recognition module, and then go through segmentation and normalization prior to translation. Excepts the machine translation module which employs mul- tilingual models, other modules use single input and output modalities corresponding to the task that they need to per- form. In this evaluation campaign, we built only sequence-to- sequence speech recognition (ASR) models with two differ- ent architectures. Similar to our previous year’ systems, the segmentation and normalization are basically a monolingual system which translates from the disfluent, broken, uncased text (i.e. ASR outputs) to a more fluent, written-style with punctuations in order to match the data conditions of the translation system. Finally, our translation system have been implemented as a multilingual system using Transformer ar- chitecture on all the data we have. Thus, it could cover all the translation directions for both sub-tasks with a little bit further fine-tuning for each target languages. This paper is structured as follows: In Section 2, we describe different speech recognition architectures we em- ployed in the campaign. Afterwards, we give a detailed de- scription of the segmentation approach in Section 3. Our multilingual machine translation system is described in Sec- tion 4. In Section 5 we report our results …

Content truncated to 500 words