PhoMT: A High-Quality and Large-Scale Benchmark Dataset for Vietnamese-English Machine Translation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4495–4503 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 4495 PhoMT: A High-Quality and Large-Scale Benchmark Dataset for Vietnamese-English Machine Translation Long Doan∗, Linh The Nguyen∗, Nguyen Luong Tran∗, Thai Hoang† and Dat Quoc Nguyen VinAI Research, Hanoi, Vietnam {v.longdct, v.linhnt140, v.nguyentl12, v.thaihq5, v.datnq9}@vinai.io Abstract We introduce a high-quality and large-scale Vietnamese-English parallel dataset of 3.02M sentence pairs, which is 2.9M pairs larger than the benchmark Vietnamese-English machine translation corpus IWSLT15. We conduct ex- periments comparing strong neural baselines and well-known automatic translation engines on our dataset and find that in both auto- matic and human evaluations: the best per- formance is obtained by fine-tuning the pre- trained sequence-to-sequence denoising auto- encoder mBART. To our best knowledge, this is the first large-scale Vietnamese-English ma- chine translation study. We hope our pub- licly available dataset and study can serve as a starting point for future research and appli- cations on Vietnamese-English machine trans- lation. We release our dataset at: https:// github.com/VinAIResearch/PhoMT. 1 Introduction Vietnam has achieved rapid economic growth in the last two decades (Baum, 2020). It is now an attrac- tive destination for trade and investment. Due to the language barrier, foreigners usually rely on au- tomatic machine translation (MT) systems to trans- late Vietnamese texts into their native language or another language they are familiar with, e.g. the global language English, so they could quickly catch up with ongoing events in Vietnam. Thus the demand for high-quality Vietnamese-English MT has rapidly increased. However, state-of-the- art MT models require high-quality and large-scale corpora for training to be able to reach near human- level translation quality (Wu et al., 2016; Ott et al., 2018). Despite being one of the most spoken lan- guages in the world with about 100M speakers, Vietnamese is referred to as a low-resource lan- guage in MT research because publicly available ∗The first three authors contributed equally to this work. †Work done during internship at VinAI Research. Email: qthai912@cs.washington.edu parallel corpora for Vietnamese in general and in particular for Vietnamese-English MT are not large enough or have low-quality translation pairs, in- cluding those with different sentence meaning (i.e. misalignment). Two main concerns are detailed as follows: • High-quality Vietnamese-English parallel cor- pora are either not publicly available or small- scale. Ngo et al. (2013) and Phan-Vu et al. (2019) present two corpora each comprising of 800K sentence pairs, however, these two corpora are not publicly available. Thus, the Vietnamese- English parallel corpus IWSLT15 (Cettolo et al., 2015) of 133K sentence pairs extracted from TED-Talks transcripts is still considered as the standard benchmark for MT when it comes to Vietnamese. Recently, the OPUS project (Tiede- mann, 2012) provides 300K+ sentence pairs ex- tracted from the TED2020 v1 corpus of TED- Talks transcripts (Reimers and Gurevych, 2020). • Larger Vietnamese-English parallel corpora are noisy, e.g. see discussions on the 300K-600K sentence pair corpora of JW300 (Agi´c and Vuli´c, 2019), OPUS’s GNOME and QED (Abdelali et al., …

Content truncated to 500 words