KC4MT: A High-Quality Corpus for Multilingual Machine Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 5494–5502 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 5494 KC4MT: A High-Quality Corpus for Multilingual Machine Translation Van-Vinh Nguyen, Ha Nguyen-Tien, Huong Le-Thanh, Phuong-Thai Nguyen, Van-Tan Bui Nghia-Luan Pham, Tuan-Anh Phan, Minh-Cong Nguyen Hoang, Hong-Viet Tran, Huu-Anh Tran Multilingual Machine Translation Project KC4.0, VNU - UET, Hanoi, Vietnam vinhnv@vnu.edu.vn, tienhapt@gmail.com, bvtan@uneti.edu.vn, huonglt@soict.hust.edu.vn, thainp@vnu.edu.vn, luanpn@dhhp.edu.vn, phantuananhkt2204k60@gmail.com, congnhm@vnu.edu.vn, thviet79@gmail.com, anhuni1006@gmail.com Abstract The multilingual parallel corpus is an important resource for many applications of natural language processing (NLP). For machine translation, the size and quality of the training corpus mainly affects the quality of the translation models. In this work, we present the method for building high-quality multilingual parallel corpus in the news domain and for some low-resource languages, including Vietnamese, Laos, and Khmer, to improve the quality of multilingual machine translation in these areas. We also publicized this one that includes 500.000 Vietnamese-Chinese bilingual sentence pairs; 150.000 Vietnamese-Laos bilingual sentence pairs, and 150.000 Vietnamese-Khmer bilingual sentence pairs. Keywords: Multilingual parallel corpus,low-resource languages, language resource, parallel corpus, machine translation 1. Introduction In recent years, with the deep learning network, mul- tilingual machine translation models have made leaps and bounds, the quality of the machine translation is close to that of a translator in some language pairs, such as French - English. Chinese-English. How- ever, state-of-the-art machine translation models re- quire high quality and large-scale parallel corpora for training to be able to reach near human-level transla- tion quality (Wu and et al., 2016). In machine translation, Vietnamese is known as a low-resource language. The public parallel corpora for Vietnamese are mainly English-Vietnamese bilin- gual corpus. Recently, A High-Quality and Large- Scale English-Vietnamese bilingual corpus was pub- lished by Vingroup, that called PhoMT (Doan et al., 2021). It includes 3.02M sentence pairs, it is very small, or even nonexistent in some language pairs, such as Vietnamese-Laos, Vietnamese-Khmer, etc. For Vietnamese, the multilingual parallel corpus is very rare, (Trieu and Ittoo, 2020) introduced a multilingual parallel corpus containing 2.5 millions parallel sen- tences on ten language pairs of several Southeast Asian languages among Filipino, Malay, Indonesian, Viet- namese. However, this corpus is not publicly available for the research community. China, Lao, Cambodia are countries bordering Viet- nam. China is Vietnam’s top trading partner, while Lao, Cambodia are two countries that have a friendly rela- tionship that is considered by the Vietnamese govern- ment as brotherly love. A good quality machine trans- lation system that translates text from Vietnamese into Chinese, Laos, or Cambodia is more essential than ever in order to support the information exchange of social- political organizations, economic groups, and people between countries. This has motivated us to focus on building a high-quality multilingual corpus, including Vietnamese, Chinese, Laos, and Cambodian languages to improve the quality of machine translation of these language pairs and publish available for the research community The rest of the paper is laid out as follows: Section 2 presents the …

Content truncated to 500 words