FGraDA: A Dataset and Benchmark for Fine-Grained Domain Adaptation in Machine Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6719–6727 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6719 FGraDA: A Dataset and Benchmark for Fine-Grained Domain Adaptation in Machine Translation Wenhao Zhu1,2, Shujian Huang1,2, Tong Pu1,2, Pingxuan Huang3, Xu Zhang4, Jian Yu4, Wei Chen4, Yanfeng Wang4, Jiajun Chen1,2 1 National Key Laboratory for Novel Software Technology, Nanjing University, China 2 Collaborative Innovation Center of Novel Software Technology and Industrialization 3 University of Michigan, USA 4 Sogou Inc. Beijing, China {zhuwh, putong}@smail.nju.edu.cn, {huangsj, chenjj}@nju.edu.cn, pxuanh@umich.edu {zhangxu216526, yujian216093, chenweibj8871, wangyanfeng}@sogou-inc.com Abstract Previous research for adapting a general neural machine translation (NMT) model into a specific domain usually neglects the diversity in translation within the same domain, which is a core problem for domain adaptation in real-world scenarios. One representative of such challenging scenarios is to deploy a translation system for a conference with a specific topic, e.g., global warming or coronavirus, where there are usually extremely less resources due to the limited schedule. To motivate wider investigation in such a scenario, we present a real-world fine-grained domain adaptation task in machine translation (FGraDA). The FGraDA dataset consists of Chinese-English translation task for four sub-domains of information technology: autonomous vehicles, AI education, real-time networks, and smart phone. Each sub-domain is equipped with a development set and test set for evaluation purposes. To be closer to reality, FGraDA does not employ any in-domain bilingual training data but provides bilingual dictionaries and wiki knowledge base, which can be easier obtained within a short time. We benchmark the fine-grained domain adaptation task and present in-depth analyses showing that there are still challenging problems to further improve the performance with heterogeneous resources. Keywords: Domain Adaptation, Fine-Grained Domains, Machine Translation 1. Introduction Recent years have witnessed the great thrive in neu- ral machine translation (Sutskever et al., 2014; Bah- danau et al., 2015; Vaswani et al., 2017). These neural- network-based models are wildly successful when there is abundant parallel data for training. However, in most real-world scenarios, the amount of data in a spe- cific domain is limited. Therefore, domain adaptation becomes a popular topic that aims at adapting transla- tion models in the general domain (or a source domain) to a target domain (Luong and Manning, 2015; Freitag and Al-Onaizan, 2016; Chu et al., 2017; Barone et al., 2017; Michel and Neubig, 2018; Vilar, 2018; Hu et al., 2019; Zhao et al., 2020). We notice that current research of domain adaptation usually considers very broad target domains. E.g., the popular dataset OPUS1 (Tiedemann, 2012) is tested for the following domains: law, medical, information technology, Koran, and subtitles (Koehn and Knowles, 2017). There are still strong diversities within each domain. For example, the subtitles domain contains subtitles from action movies, political movies, Sci-Fi movies, etc. We suggest that there are fine-grained sub-domains within these coarse domains. The sentences or words in different sub-domains may have different language phenomena, which requires a fine-grained treatment. As shown in Table 1, …

Content truncated to 500 words