Better Translation + Split and Generate for Multilingual RDF-to-Text (WebNLG 2023)

Proceedings of the Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge (MM-NLG 2023), pages 73–79 September 12, 2023 ©2023 Association for Computational Linguistics Better Translation + Split and Generate for Multilingual RDF-to-Text Nalin Kumar♣, Saad Obaid ul Islam♠, Ondˇrej Dušek♣ ♣Charles University, Faculty of Mathematics and Physics, Prague, Czechia ♠Center For Artificial Intelligence and Data Science, University of Würzburg, Germany kumarnal@student.cuni.cz, saad.obaid-ul-islam@uni-wuerzburg.de, odusek@ufal.mff.cuni.cz Abstract This paper presents system descriptions of our submitted outputs for WebNLG Challenge 2023. We use mT5 in multi-task and multilin- gual settings to generate more fluent and re- liable verbalizations of the given RDF triples. Furthermore, we introduce a partial decoding technique to produce more elaborate yet sim- plified outputs. Additionally, we demonstrate the significance of employing better translation systems in creating training data. 1 Introduction State-of-the-art in natural language generation (NLG) recently has benefited from self-supervised pretrained language models (PLMs) such as BART, T5, and GPT-2 (Lewis et al., 2020; Raffel et al., 2020; Radford et al., 2019), which provided perfor- mance improvements on many NLG tasks (Zhang et al., 2020b; Wei et al., 2021; Kale and Rastogi, 2020). One of the most prominent NLG tasks is data-to-text generation (Wiseman et al., 2017), with multiple public datasets, including WebNLG (Colin et al., 2016), ToTTo (Parikh et al., 2020), or DART (Nan et al., 2021). A significant prob- lem of these datasets is their limitation to English only, which effectively rules out their application to other languages; incorporating low-resource languages in NLG is generally an open problem (Ruder, 2019). Researchers addressed this by multilingual PLMs like mT5 (Xue et al., 2021) and mBART (Liu et al., 2020) pre-trained on massively multi- lingual corpora like Common Crawl 101.1 Unfortu- nately, given the disproportionate representation of low-resource languages in multilingual datasets,2 the efficacy of PLMs on these languages is com- promised. Efforts are being made to improve PLM 1https://commoncrawl.org/ 2https://commoncrawl.github.io/ cc-crawl-statistics/plots/languages performance for low-resource languages via cross- lingual transfer (Conneau et al., 2020). To assess the state-of-the-art performance and spur devel- opment in data-to-text NLG techniques for low- resource languages, the WebNLG 2023 challenge features five non-English languages – four entirely new low-resource ones: Maltese (Mt), Welsh (Cy), Irish (Ga) and Breton (Br), with the addition of Russian (Ru) from the 2020 iteration (Castro Fer- reira et al., 2020). While development and test sets for the low-resource languages were created by hand, training sets are machine-translated. In our submission of WebNLG 2023 challenge, we use the power of mT5 (Xue et al., 2021), a mul- tilingual transformer model trained in a multi-task fashion. To boost performance on low-resource lan- guages, we employ several additional steps in fine- tuning and inference: (1) We improve WebNLG 2023 training data by re-translating with a stronger machine translation (MT) model (Costa-jussà et al., 2022). (2) We finetune either individual models for each language (single-task setting), or a single model for all languages (multi-task setting). (3) We generate text by splitting the input RDF triples and decoding a subset of …

Content truncated to 500 words