High Frequent In-domain Words Segmentation and Forward Translation for the WMT21 Biomedical Task

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 863–867 November 10–11, 2021. ©2021 Association for Computational Linguistics 863 High frequent in-domain word segmentation and forward translation for the WMT21 Biomedical task Bardia Rafieian and Marta R. Costa-jussà TALP Research Center, Universitat Politècnica de Catalunya, Barcelona {bardia.rafieian,marta.ruiz}@upc.edu Abstract This paper reports the optimization of us- 001 ing the out-of-domain data in the Biomedi- 002 cal translation task. We firstly optimized our 003 parallel training dataset using the BabelNet 004 in-domain terminology words. Afterward, to 005 increase the training set, we studied the ef- 006 fects of the out-of-domain data on biomedi- 007 cal translation tasks, and we created a mixture 008 of in-domain and out-of-domain training sets 009 and added more in-domain data using forward 010 translation in the English-Spanish task. Fi- 011 nally, with a simple bpe optimization method, 012 we increased the number of in-domain sub- 013 words in our mixed training set and trained the 014 Transformer model on the generated data. Re- 015 sults show improvements using our proposed 016 method. 017 1 Introduction 018 Domain adaptation is one of the known challenges 019 in Machine Translation since NMT(neural machine 020 translation) models are susceptible to the training 021 data (Koehn and Knowles, 2017). To say, NMT 022 models perform poorly for domain-specific transla- 023 tion when trained on large out-resource data (Chu 024 and Wang, 2018). As a result, due to the limita- 025 tions of specific domain data, domain adaptation 026 strategies help NMT models by increasing the par- 027 allel corpora. There have been several tasks to 028 address domain adaptation which recently, in (Sato 029 et al., 2020) they proposed a vocabulary adapta- 030 tion to fine-tune the embedding layers of the NMT 031 model by projecting general word embeddings in- 032 duced from monolingual data in a target domain 033 onto a source-domain embedding space to improve 034 translation score. On the other hand, augmenting 035 bilingual training data with forwarding and back- 036 ward translation improves the in-domain translation 037 quality (Nayak et al., 2020). Inspired by mentioned 038 ideas, in this work, we implemented our strategy by 039 two essential steps: 1) collecting and augmenting 040 data by forwarding translation and then tuning it 041 using Babelnet to include biomedical sentences 2) 042 Implementing subwords bpe optimization on the 043 train set to study the adaptation of out-of-domain 044 data in the biomedical task. After that, We selected 045 the transformer model (Vaswani et al., 2017) to 046 train our system in different experimental settings. 047 The remainder of the paper is organized as follows. 048 In Sec. 2 we describe data collection and prepara- 049 tion. Sec. 3 explains our bpe optimization strategy 050 to adapt out-of-domain data in the biomedical task. 051 Sec. 4 shows our experimental setups and evalua- 052 tion results, and finally, we conclude and discuss 053 future works in the Sec. 5. 054 2 Data production 055 One of the critical topics …

Content truncated to 500 words