JUST System for WMT20 Chat Translation Task

Proceedings of the 5th Conference on Machine Translation (WMT), pages 479–482 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 479 JUST System for WMT20 Chat Translation Task Roweida Mohammed, Mahmoud Al-Ayyoub and Malak Abdullah Jordan University of Science and Technology Irbid, Jordan roweida.221@gmail.com,{maalshbool, mabdullah}@just.edu.jo Abstract Machine Translation (MT) is a sub-field of Artificial Intelligence and Natural Language Processing that investigates and studies the ways of automatically translating a text from one language to another. In this paper, we present the details of our submission to the WMT20 Chat Translation Task, which consists of two language directions, English→German and German→English. The major feature of our system is applying a pre-trained BERT em- bedding with a bidirectional recurrent neural network. Our system ensembles three mod- els, each with different hyperparameters. De- spite being trained on a very small corpus, our model produces surprisingly good results. 1 Introduction The language of chat texts is considered a common language where people are rarely paying attention to correct spelling. Therefore, using the traditional methods of Machine Translation (MT), like dictio- naries, is insufficient (Hern´andez, 2009). As deep learning (DL) models are becoming more evolved and complex, this motivates the natural language processing (NLP) community researchers to em- ploy them for challenging tasks such as MT of informal language, such as what is used in chat. Techniques like contextual word embeddings and pre-trained DL models are becoming very common in natural language generation (NLG) tasks such as MT (Kusner et al., 2015; Zou et al., 2013; Abdullah and Shaikh, 2018; Al-Bdour et al., 2019). The Chat Translation Task is a new task in the Fifth Conference on Machine Translation (WMT20).1 Translating chat text, specifically the chats of customer support, is a main and exciting task in the field of MT. This kind of tasks has not been widely considered in previous MT studies, 1http://www.statmt.org/wmt20/chat-task.html mostly because of the absence of openly existing datasets. The target of this new Chat Translation Task is to translate the customer support chat text from English to German and vice versa. The essen- tial goal of this task is to develop models that can translate conversational text and study the use of multilingual models. We take part in the WMT20 shared chat translation task in two language directions: English→German and German→English. In this paper, we discuss our submission for this task, which is based on the bidirectional recurrent neu- ral networks (bi-RNN) (Schuster and Paliwal, 1997) and using the pre-trained BERT embed- ding, known as bert-base-multilingual-cased (De- vlin et al., 2018). This paper is constructed as follows. In Sec- tion 2, the task and data descriptions are provided. Section 3 discusses our proposed model. Section 4 shows the experiments we conduct and their results. Finally, the Conclusion is in Section 5. 2 Task and Data Description The Chat Translation shared task of WMT20 offers participants the opportunity to address a challeng- ing problem faced by many companies today as they expand their customer support units to multi- ple different languages. …

Content truncated to 500 words