Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1072–1086 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 1072 Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints Zhenyi Wang†§∗, Xiaoyang Wang§, Bang An†, Dong Yu§, Changyou Chen† †State University of New York at Buffalo, §Tencent AI Lab, Bellevue, WA {zhenyiwa, anbang, changyou}@buffalo.edu {shawnxywang, dyu}@tencent.com Abstract Text generation from a knowledge base aims to translate knowledge triples to natural- language descriptions. Most existing methods ignore the faithfulness between a generated text description and the original table, lead- ing to generated information that goes beyond the content of the table. In this paper, for the first time, we propose a novel Transformer- based generation framework to achieve the goal. The core techniques in our method to enforce faithfulness include a new table-text optimal-transport matching loss and a table- text embedding similarity loss based on the Transformer model. Furthermore, to evaluate faithfulness, we propose a new automatic met- ric specialized to the table-to-text generation problem. We also provide detailed analysis on each component of our model in our ex- periments. Automatic and human evaluations show that our framework can significantly out- perform state-of-the-art by a large margin. 1 Introduction Understanding structured knowledge, e.g., infor- mation encoded in tables, and automatically gen- erating natural-language descriptions is an impor- tant task in the area of Natural Language Gen- eration. Table-to-text generation helps making knowledge elements and their connections in ta- bles easier to comprehend by human. There have been a number of practical application scenarios in this field, for example, weather report genera- tion, NBA news generation, biography generation and medical-record description generation (Liang et al., 2009; Barzilay and Lapata, 2005; Lebret et al., 2016a; Cawsey et al., 1997). Most existing methods for table-to-text gen- eration are based on an encoder-decoder frame- work (Sutskever et al., 2014; Bahdanau et al., ∗Zhenyi Wang was a research intern student at Tencent AI Lab in Bellevue, WA when doing this work. Figure 1: An example of table-to-text generation. This generation is unfaithful because there exists informa- tion in table not covered by generated text (marked in blue); At the same time, hallucinated information in text does not appear in table (marked in red). 2015), most of which are RNN-based Sequence- to-Sequence (Seq2Seq) models (Lebret et al., 2016b; Liu et al., 2018; Wiseman et al., 2018; Ma et al., 2019; Wang et al., 2018; Liu et al., 2019a). Though significant progress has been achieved, we advocate two key problems in existing methods. Firstly, because of the intrinsic shortage of RNN, RNN-based models are not able to capture long- term dependencies, which would lose important information reflected in a table. This drawback prevents them from being applied to larger tables, for example, a table describing a large Knowledge Base (Wang et al., 2018). Secondly, little work has focused on generating faithful text descrip- tions, which is defined, in this paper, as the level of matching between a generated text sequence 1073 and the corresponding …

Content truncated to 500 words