Kakao Enterprise’s WMT21 Machine Translation Using Terminologies Task Submission

Proceedings of the Sixth Conference on Machine Translation (WMT), pages 804–812 November 10–11, 2021. ©2021 Association for Computational Linguistics 804 Kakao Enterprise’s WMT21 Machine Translation using Terminologies Task Submission Yunju Bak1 Jimin Sun2* Jay Kim1 Sungwon Lyu1 Changmin Lee1 1Kakao Enterprise 2Carnegie Mellon University 1{juliette.y,jay.ka387,james.ryu,louis.cm}@kakaoenterprise.com 2jimins2@cs.cmu.edu Abstract This paper describes Kakao Enterprise’s sub- mission to the WMT21 shared Machine Trans- lation using Terminologies task. We integrate terminology constraints by pre-training with target lemma annotations and fine-tuning with exact target annotations utilizing the given terminology dataset. This approach yields a model that achieves outstanding results in terms of both translation quality and term con- sistency, ranking first based on COMET in the En→Fr language direction. Furthermore, we explore various methods such as back- translation, explicitly training terminologies as additional parallel data, and in-domain data se- lection. 1 Introduction We participate in the WMT21 Machine Trans- lation using Terminologies Task in four lan- guage directions, English→French (En→Fr), English→Chinese (En→Zh), English→Korean (En→Ko) and Czech→German (Cs→De). 1.1 Task description The recent COVID-19 pandemic has raised the ur- gency to translate and distribute the latest medical information worldwide. However, despite recent ad- vances in neural machine translation (NMT), trans- lation in such emerging domains remains a chal- lenge, as it is unaffordable to collect fair amounts of quality in-domain parallel data in a short time. As an alternative, word- or phrase-level dictionaries of key terms are relatively easier to obtain. These dictionaries are prevalent in commercial settings, where customers specify domain-specific jargon that human translators can attend to. However, in- corporating pre-specified dictionaries effectively into NMT models is a non-trivial problem, as NMT *Work done during the author’s internship at Kakao En- terprise. is inherently trained without explicit constraints compared to statistical approaches. In this context, the shared task of Machine Trans- lation using Terminologies is held in five language directions at WMT21. The task assumes a realis- tic scenario where parallel and monolingual data are abundant in generic domains (e.g., news, web crawl), but only hundreds of word- or phrase-level term dictionaries are available in the domain of in- terest — COVID-19. Technically, this poses a chal- lenge as we must impose terminology constraints without hurting general translation quality, while only 1.5% of parallel data contain the provided ter- minologies. Additional issues such as the 1 : N mapping of term translations further complicate the problem. Evaluating MT systems in specialized domains diverge from general MT evaluation in that overall translation quality may not ensure the translation accuracy of domain-specific terms. This potential gap calls the need for evaluation metrics that di- rectly assess the consistent use of terms. Concretely, three metrics proposed in Alam et al. (2021) are employed in this task – Exact-Match Accuracy, Window Overlap, and Terminology-biased Trans- lation Edit Rate (TERm). The suggested metrics complement general translation accuracy measured by standard MT metrics (BLEU, chrF, BERTscore, COMET) by validating whether terms are trans- lated faithfully according to the dictionary. Specifically, human-labeled COVID-19 related term dictionaries are released in four language …

Content truncated to 500 words