Improving User Controlled Table-To-Text Generation Robustness

Findings of the Association for Computational Linguistics: EACL 2023, pages 2317–2324 May 2-6, 2023 ©2023 Association for Computational Linguistics Improving User Controlled Table-To-Text Generation Robustness Hanxu Hu1 , Yunqing Liu2, Zhongyi Yu1 and Laura Perez-Beltrachini 1 1 School of Informatics, University of Edinburgh, United Kingdom 2 The Hong Kong Polytechnic University, HongKong {huhanxu1233,lyq6175215241,zhongyics}@gmail.com lperez@exseed.ed.ac.uk Abstract In this work we study user controlled table-to- text generation where users explore the con- tent in a table by selecting cells and reading a natural language description thereof automat- ically produce by a natural language genera- tor. Such generation models usually learn from carefully selected cell combinations (clean cell selections); however, in practice users may se- lect unexpected, redundant, or incoherent cell combinations (noisy cell selections). In exper- iments, we find that models perform well on test sets coming from the same distribution as the train data but their performance drops when evaluated on realistic noisy user inputs. We propose a fine-tuning regime with additional user-simulated noisy cell selections. Models fine-tuned with the proposed regime gain 4.85 BLEU points on user noisy test cases and 1.4 on clean test cases; and achieve comparable state- of-the-art performance on the ToTTo dataset.1 1 Introduction The goal of table-to-text generation is to provide the user with a description of the most relevant con- tent in a given table (Lebret et al., 2016; Wiseman et al., 2018; Perez-Beltrachini and Lapata, 2018; Puduppully et al., 2019). Recently, Parikh et al. (2020) proposed a controlled table-to-text genera- tion task where the goal is to automatically create a description for a determined subset of the table, namely the highlighted table cells. The main fo- cus on Parikh et al.’s 2020 work is to assess the performance of neural text generators in a more controlled setting, i.e., when given an input table with explicit instructions (i.e., highlights) on what should be expressed in the output description. In this work, we view this task in the context of a nat- ural language interface, as a user controlled table- to-text generation task, where users provide those 1Our code is available at https://github.com/hanxuhu/controllT2Trobust Figure 1: An example in the ToTTo dataset. The figure is retrieved from (Parikh et al., 2020). The cells coloured in yellow are the highlight cells. highlights interactively by exploring the content of a given table and study these user interactions. Figure 1 illustrates the case where a user selects some cells (highlighted in yellow) and the gener- ator provides a description thereof (shown below the table). A crucial aspect of usability assessment for a generator in this interactive table-to-text task is ro- bustness. In a recent study by (Mille et al., 2021), it has been shown that neural generation models fail to maintain their in distribution performance when confronted with realistic scenarios at test time such as typos in the input text. In the case of user controlled table-to-text generation, users may in- troduce noise when exploring the table content and select cell combinations that turn out to be unex- pected, …

Content truncated to 500 words