JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3603–3609 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3603 JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus Makoto Morishita, Jun Suzuki, and Masaaki Nagata NTT Communication Science Laboratories, NTT Corporation 2-4 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0237, Japan {makoto.morishita.gr, masaaki.nagata.et}@hco.ntt.co.jp jun.suzuki@ecei.tohoku.ac.jp Abstract Recent machine translation algorithms mainly rely on parallel corpora. However, since the availability of parallel corpora remains limited, only some resource-rich language pairs can benefit from them. We constructed a parallel corpus for English-Japanese, for which the amount of publicly available parallel corpora is still limited. We constructed the parallel corpus by broadly crawling the web and automatically aligning parallel sentences. Our collected corpus, called JParaCrawl, amassed over 8.7 million sentence pairs. We show how it includes a broader range of domains and how a neural machine translation model trained with it works as a good pre-trained model for fine-tuning specific domains. The pre-training and fine-tuning approaches achieved or surpassed performance comparable to model training from the initial state and reduced the training time. Additionally, we trained the model with an in-domain dataset and JParaCrawl to show how we achieved the best performance with them. JParaCrawl and the pre-trained models are freely available online for research purposes. Keywords: parallel corpus, machine translation, English, Japanese 1. Introduction Since current machine translation (MT) approaches are mainly data-driven, one key bottleneck has been the lack of parallel corpora. This problem continues with the re- cent neural machine translation (NMT) architecture. As the amount of training data increases, NMT performance improves (Sennrich and Zhang, 2019). Our goal is to create large parallel corpora to/from Japanese. In our first attempt, we focused on the English-Japanese language pair. Currently, ASPEC is the largest publicly available English-Japanese parallel corpus (Nakazawa et al., 2016), which contains 3.0 million sen- tences for training. Unfortunately, this is relatively small compared to such resource-rich language pairs as French- English1. Also, available domains remain limited. We ad- dress this problem, which hinders the progress of English- Japanese translation research, by crawling the web to mine for English-Japanese parallel sentences. Current NMT training requires a great deal of computa- tional time, which complicates running experiments with few computational resources. We alleviate this problem by providing NMT models trained with our corpus. Since our web-based parallel corpus contains a broad range of do- mains, it might be used as a pre-trained model and fine- tuned with a domain-specific parallel corpus. The following are the contributions of this paper: • We released the largest freely available web-based English-Japanese parallel corpus: JParaCrawl. • We also released NMT models trained with JParaCrawl for further fine-tuning. The rest of the paper is organized as follows. In Section 2, we introduce previous work that tried to create parallel cor- pora by mining parallel texts. In Section 3, we show how we created JParaCrawl. We conducted experiments to show 1Currently the largest French-English parallel corpus is ParaCrawl v5, which contains …

Content truncated to 500 words