De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5532–5542 August 1–6, 2021. ©2021 Association for Computational Linguistics 5532 De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation Wenqing Chen1,2, Jidong Tian1,2, Yitian Li1,2, Hao He1,2∗, Yaohui Jin1,2∗ 1MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University 2State Key Lab of Advanced Optical Communication System and Network, Shanghai Jiao Tong University {wenqingchen, frank92, yitian_li, hehao, jinyh}@sjtu.edu.cn Abstract Logical table-to-text generation aims to auto- matically generate fluent and logically faithful text from tables. The task remains challenging where deep learning models often generated linguistically fluent but logically inconsistent text. The underlying reason may be that deep learning models often capture surface-level spurious correlations rather than the causal re- lationships between the table x and the sen- tence y. Specifically, in the training stage, a model can get a low empirical loss with- out understanding x and use spurious statis- tical cues instead. In this paper, we propose a de-confounded variational encoder-decoder (DCVED) based on causal intervention, learn- ing the objective p(y|do(x)). Firstly, we pro- pose to use variational inference to estimate the confounders in the latent space and co- operate with the causal intervention based on Pearl’s do-calculus to alleviate the spurious correlations. Secondly, to make the latent confounder meaningful, we propose a back- prediction process to predict the not-used en- tities but linguistically similar to the exactly selected ones. Finally, since our variational model can generate multiple candidates, we train a table-text selector to find out the best candidate sentence for the given table. An extensive set of experiments show that our model outperforms the baselines and achieves new state-of-the-art performance on two logi- cal table-to-text datasets in terms of logical fi- delity. 1 Introduction Data-to-text generation refers to the task of gen- erating descriptive text from non-linguistic inputs. With the different types of inputs, this task can be ∗ Corresponding Authors defined more specifically, such as abstract mean- ing representation to text (Zhao et al., 2020; Bai et al., 2020a), infobox with key-value pairs to text (Bai et al., 2020b), graph-to-text (Song et al., 2020), and table-to-text (Wang et al., 2020; Parikh et al., 2020) generation. Among these tasks, we focus on logical table- to-text generation, which aims to generate fluent and logically faithful text from tables (Chen et al., 2020a). And the ability of logical inference is a kind of high-level intelligence, which is non- trivial for text generation systems in reality. The task remains challenging because the reference sentences often convey logically inferred informa- tion, which is not explicitly presented in the table. As a consequence, data-driven models often gener- ated linguistically fluent but logically inconsistent text. Recent progress on this task mainly lies in the use of pretrained language models (LMs) like GPT-2 (Radford et al., 2018), which was shown to perform much better than non-pretrained models (Chen et al., 2020a,e). However, it is still arguable that whether pre- trained LMs …

Content truncated to 500 words