YerevaNN’s Systems for WMT20 Biomedical Translation Task: The Effect of Fixing Misaligned Sentence Pairs

Proceedings of the 5th Conference on Machine Translation (WMT), pages 820–825 Online, November 19–20, 2020. c⃝2020 Association for Computational Linguistics 820 YerevaNN’s Systems for WMT20 Biomedical Translation Task: The Effect of Fixing Misaligned Sentence Pairs Karen Hambardzumyan1, Hovhannes Tamoyan1,2, and Hrant Khachatrian1,3 1YerevaNN 2American University of Armenia 3Department of Informatics and Applied Mathematics, Yerevan State University Abstract This report describes YerevaNN’s neural machine translation systems and data processing pipelines developed for WMT20 biomedical translation task. We provide systems for English-Russian and English- German language pairs. For the English- Russian pair, our submissions achieve the best BLEU scores, with en→ru direction outperforming the other systems by a significant margin. We explain most of the improvements by our heavy data preprocessing pipeline which attempts to fix poorly aligned sentences in the parallel data. 1 Introduction Biomedical machine translation is a perfect playground to develop narrow domain neural machine translation models. In such tasks, the available parallel in-domain data is usually limited and noisy which creates many challenges. In the previous works (Bawden et al., 2019), researchers focused on transfer learning methods (Saunders et al., 2019) or attempted to mix the training data with other sources (Peng et al., 2019) to address the issue of data scarcity. In this work, we show that the transfer performance is very dependent on the quality of the training data, and with a little effort, it is possible to improve the given MEDLINE training data and gain a significant performance boost. We have manually created a much higher quality subset of the original MEDLINE training data for local evaluation purposes. The insights collected during this manual analysis was then used to fix the most common issues within the training data. In particular, we noticed that the original dataset contained paper abstracts in two languages without sentence-level alignments, and the training corpus provided by the organizers was created using an automated sentence segmentation and alignment process, which was not perfect. We built a data pipeline1 that handles 1) cleanup, 2) sentence segmentation, 3) alignment of translation sentence pairs and 4) preprocessing. In our experiments, we did not use any data source other than MEDLINE. We chose our baseline model and two other models with the highest BLEU scores on a local test set as our three submissions. The best ones got 35.2% BLEU on English-German and 41.3% on German-English test sets. For English-Russian and Russian-English directions we reached BLEU scores of 37.9% and 43.2% respectively, which are the best scores among all submissions of WMT20 Biomedical Translation Task. Moreover, our models are cheap to train: the average training time of our best models is approximately 30 minutes on a single NVIDIA Titan V GPU. The paper is organized as follows: Section 2 presents fine-tuning details and evaluation methods for our NMT systems, Section 3 describes the data used in the experiments and the data processing pipeline, Section 4 presents our novel method of monotonic alignment based on multilingual language models. Section 5 discusses the results. 2 System Description …

Content truncated to 500 words