ParaZh-22M: A Large-Scale Chinese Parabank via Machine Translation

Proceedings of the 29th International Conference on Computational Linguistics, pages 3885–3897 October 12–17, 2022. 3885 ParaZh-22M: a Large-Scale Chinese Parabank via Machine Translation Wenjie Hao1, Hongfei Xu1, Deyi Xiong2, Hongying Zan1,3, Lingling Mu1∗ 1 Zhengzhou University, 2 Tianjin University, 3 Peng Cheng Laboratory haowj9977@163.com, hfxunlp@foxmail.com, dyxiong@tju.edu.cn {iehyzan,iellmu}@zzu.edu.cn Abstract Paraphrasing, i.e., restating the same meaning in different ways, is an important data augmen- tation approach for natural language process- ing (NLP). Zhang et al. (2019b) propose to extract sentence-level paraphrases from mul- tiple Chinese translations of the same source texts, and construct the PKU Paraphrase Bank of 0.5M sentence pairs. However, despite being the largest Chinese parabank to date, the size of PKU parabank is limited by the availability of one-to-many sentence translation data, and cannot well support the training of large Chi- nese paraphrasers. In this paper, we relieve the restriction with one-to-many sentence transla- tion data, and construct ParaZh-22M, a larger Chinese parabank that is composed of 22M sentence pairs, based on one-to-one bilingual sentence translation data and machine trans- lation (MT). In our data augmentation exper- iments, we show that paraphrasing based on ParaZh-22M can bring about consistent and sig- nificant improvements over several strong base- lines on a wide range of Chinese NLP tasks, in- cluding a number of Chinese natural language understanding benchmarks (CLUE) and low- resource machine translation. 1 1 Introduction A paraphrase is a restatement of meaning with dif- ferent expressions (Bhagat and Hovy, 2013). Para- phrasing has been proven to be an effective data augmentation approach for many NLP tasks, rang- ing from linguistically controlled paraphrase gener- ation (Iyyer et al., 2018; Chen et al., 2019; Li et al., 2019; Sun et al., 2021), style transfer (Krishna et al., 2020), to applications like low-resource machine translation (Khayrallah et al., 2020) and automatic MT evaluation (Thompson and Post, 2020; Bawden et al., 2020). ∗ Corresponding author. 1We opensource our dataset at https://github. com/haowj9977/parazh-22M. Zhang et al. (2019b) extract sentence-level para- phrases from multiple Chinese translations of the same source texts, and create the largest Chinese paraphrase bank (PKU Parabank) to date, which contains 509,832 pairs of paraphrased sentences. However, the amount of one-to-many sentence translation data constrains the size of PKU para- bank, and it cannot meet the requirement to train large Chinese paraphrasers. Inspired by Wieting and Gimpel (2018) and Hu et al. (2019a,b), we propose to relax the restric- tion that requires one-to-many translation data on the construction of large-scale Chinese parabanks, by utilizing bilingual one-to-one translation data of larger scales and MT, and construct ParaZh- 22M. Specifically, we leverage the huge Chinese- English machine translation data from WMT 2021 (Akhbardeh et al., 2021) of 30.4M sentence pairs, apply strict rules to ensure the data quality, and translate the English side of the parallel corpus to Chinese with the cutting-edge deep Transformers and several approaches to ensure the translation quality. We pick the machine translated Chinese sentences considering both diversity and semantic consistency, and pair with the corresponding orig- inal Chinese references …

Content truncated to 500 words