WebNLG-Interno: Utilizing FRED-T5 to address the RDF-to-text problem (WebNLG 2023)

Proceedings of the Workshop on Multimodal, Multilingual Natural Language Generation and Multilingual WebNLG Challenge (MM-NLG 2023), pages 67–72 September 12, 2023 ©2023 Association for Computational Linguistics WebNLG-Interno: Utilizing FRED-T5 to address the RDF-to-text problem Maxim Kazakov Petal Cloud Technology Co.,Ltd HSE University kazakov.maxim@huawei.com Julia Preobrazhenskaya Lobachevsky State University enjulia17@gmail.com Ivan Bulychev Lobachevsky State University halkimic@gmail.com Aleksandr Shain Petal Cloud Technology Co.,Ltd Pulkovo Observatory alexander.shain@huawei.com Abstract We present our solution for the Russian RDF- to-text generation task of the WebNLG Chal- lenge 2023 1. We use the pretrained large language model named FRED-T5 (Zmitrovich et al., 2023) to finetune on the train dataset. Also, we propose several types of prompt and run experiments to analyze their effectiveness. Our submission achieves 0.373 TER on the test dataset, taking the first place according to the results of the automatic evaluation and outperforming the best result of the previous challenge by 0.025. The code of our solu- tion is available at the following link: https: //github.com/Ivan30003/webnlg_interno 1 Introduction Recently released large language models (a.k.a. LLMs) like GPT-4 (OpenAI, 2023), BLOOM (Scao et al., 2022), LLaMA (Touvron et al., 2023) and PaLM (Chowdhery et al., 2022) proved the abil- ity of deep neural networks to generate realistic texts, maintain a human-like conversation and an- swer factual questions based on the information contained in the training data. However, real-life applications of LLMs could benefit from extracting relevant information from external databases to pro- vide a user with a proper answer. Data in databases typically have a structured form, and one of the main challenges is to present extracted information in the natural sentences. The RDF-to-text track of the WebNLG challenge aims to address that particular problem. Given the data presented in a common RDF format, the task is to generate a natural utterance that conveys in- formation from the structured data. The RDF data 1https://synalp.gitlabpages.inria.fr/ webnlg-challenge/challenge_2023 format operates with three entities - subject, object and predicate, where the latter represents a type of relations between subject and object. Such entities form a triple, and each data sample is represented as a tripleset consisting of one or several triples (Table 1). It was mentioned by Kasner and Dušek (2020), such tripleset representation in English can be seen as a noisy version of a target utterance, and de- noising autoencoders like T5 (Raffel et al., 2020) and BART (Lewis et al., 2020) may provide a sta- ble solution to the problem. In Russian track of the challenge we additionally expect generated ut- terances to be in Russian while the triples remain in English. That can be handled either by a two stage solution where translation (of input triples or generated utterances) performed as a separate step, or by an end-to-end solution - for the such case, encoder-decoder architectures are suitable as well (Liu et al., 2020). The above assumptions are supported by the re- sults of the previous challenge (Castro Ferreira et al., 2020), where 3 out of 6 solutions for the RDF-to-text Russian track were based on …

Content truncated to 500 words