Improving Machine Translation with Phrase Pair Injection and Corpus Filtering

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5395 - 5400 December 7-11, 2022 ©2022 Association for Computational Linguistics Improving Machine Translation with Phrase Pair Injection and Corpus Filtering Akshay Batheja, Pushpak Bhattacharyya CFILT, Indian Institute of Technology Bombay {akshaybatheja, pb}@cse.iitb.ac.in Abstract In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation (NMT) systems. We extract par- allel phrases and sentences from the pseudo- parallel corpus and augment it with the parallel corpus to train the NMT models. With the proposed approach, we observe an improve- ment in the Machine Translation (MT) sys- tem for 3 low-resource language pairs, Hindi- Marathi, English-Marathi, and English-Pashto, and 6 translation directions by up to 2.7 BLEU points, on the FLORES test data. These BLEU score improvements are over the models trained using the whole pseudo-parallel corpus aug- mented with the parallel corpus. 1 Introduction Deep Neural Architectures are data hungry. It is believed that the robustness of an NMT model de- pends on the size of the training corpus. However, not all language pairs have a substantial amount of parallel data. The primary data resource for low-resource languages is the web. However, the web-crawled data contains a lot of noise that de- grades the performance of NMT systems. Hence, the quality of the training data is as important as its quantity. We aim to improve the quality of Machine Translation for Hindi-Marathi, English- Marathi, and English-Pashto language pairs by us- ing the LaBSE-based (Feng et al., 2020) corpus filtering along with the Phrase Pair Injection. We use Phrase Pair Injection to increase the size and LaBSE-based corpus filtering to improve the qual- ity of the parallel data extracted from the pseudo- parallel corpus. We observe that using these two techniques together makes the optimum use of the pseudo-parallel corpus. The contributions of this work are as follows: • We extract good quality parallel sentences and phrases from the pseudo-parallel corpus of huge size. To the best of our knowledge, this is the first method that combines Phrase Pair Injection with neural-based Corpus Filtering and extracts both good quality parallel sen- tences and phrases from the pseudo-parallel corpus. • We show that the extracted parallel sentences and phrases significantly improve the perfor- mance of the MT systems. 2 Related Work Neural Networks have become very popular with increased computational power in recent times. The Transformer model introduced by Vaswani et al. (2017) gave significant improvements in the quality of translation as compared to the previous approaches (Bahdanau et al., 2015; Sutskever et al., 2014). Transformers allow parallelization, which enables the model to train faster and get better per- formances. There are several prior studies to improve the quality and size of the parallel corpus. A set of heuristic rules were proposed by Lu et al. (2020) to remove low-quality sentence pairs from the noisy parallel corpus. Another popular approach is to compute cosine similarities between the source …

Content truncated to 500 words