A Benchmark Dataset for Multi-Level Complexity-Controllable Machine Translation

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 6744–6752 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 6744 A Benchmark Dataset for Multi-Level Complexity-Controllable Machine Translation Kazuki Tani1, Ryoya Yuasa1, Kazuki Takikawa2, Akihiro Tamura1, Tomoyuki Kajiwara2, Takashi Ninomiya2, Tsuneo Kato1 1Doshisha University, 2Ehime University {ctwh0176@mail4, ctwh0190@mail4, aktamura@mail, tsukato@mail}.doshisha.ac.jp {takikawa@ai., kajiwara@, ninomiya@}cs.ehime-u.ac.jp Abstract This paper introduces a new benchmark test dataset for multi-level complexity-controllable machine translation (MLCC-MT), which is an MT that controls the output complexity at more than two levels. In previous studies, MLCC-MT models have been evaluated on a test dataset automatically generated from the Newsela corpus, which is a document-level comparable corpus with document-level complexity. There are three issues with the existing test dataset: first, a source language sentence and its target language sentence are not necessarily an exact translation pair because they are automatically detected. Second, a target language sentence and its simplified target language sentence are not always perfectly parallel since they are automatically aligned. Third, a sentence-level complexity is not always appropriate because it is derived from an article-level complexity associated with the Newsela corpus. Therefore, we created a benchmark test dataset for Japanese-to-English MLCC-MT from the Newsela corpus by introducing an automatic filtering of data with inappropriate sentence-level complexity, manual check for parallel target language sentences with different complexity levels, and manual translation. Furthermore, we implement two MLCC-NMT frameworks with a Transformer architecture and report their performance on our test dataset as baselines for future research. Our test dataset and codes are released. Keywords: Machine Translation, Natural Language Generation, Corpus (Creation, Annotation, etc.) 1. Introduction In recent years, neural machine translation (NMT), which translates an input sentence (a source language sentence) into a sentence in the target language (a target language sentence) using neural networks, has become increasingly developed and widespread to a wide range of users. While conventional NMT uniformly gener- ates a target language sentence for any user and any situation, style-controllable NMT, which controls the style of output rather than its content, has recently re- ceived a lot of attention. Sennrich et al. (2016), for ex- ample, proposed an NMT model that controls the po- liteness of a target language sentence through the in- corporation of special tokens representing politeness, and Kuczmarski et al. (2018) proposed a method to control the gender of pronouns in target language sen- tences. Schioppa et al. (2021) proposed a method to control multiple attributes of MT outputs. Furthermore, complexity-controllable NMT (CC-NMT), which con- trols the complexity of a target language sentence to allow translation tailored to the user’s reading level, has been attracting much attention. Most of the exist- ing CC-NMT models control the complexity of a tar- get language sentence at two levels (i.e., complex and simple), which we refer to as two-level complexity- controllable NMT (2LCC-NMT). In the previous re- search, English-European language pairs have been primarily focused on. For example, Marchisio et al. (2019) have proposed a 2LCC-NMT model between Spanish and …

Content truncated to 500 words