Program Enhanced Fact Verification with Verbalization and Graph Attention Network

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7810–7825, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7810 Program Enhanced Fact Verification with Verbalization and Graph Attention Network Xiaoyu Yang†∗, Feng Nie§∗, Yufei Feng†, Quan Liu‡, Zhigang Chen‡, Xiaodan Zhu† † ECE & Ingenuity Labs Research Institute, Queen’s University § Sun Yat-sen University ‡ State Key Laboratory of Cognitive Intelligence, iFLYTEK Research {xiaoyu.yang, xiaodan.zhu, yufei.feng}@queensu.ca fengniesysu@gmail.com {quanliu, zgchen}@iflytek.com Abstract Performing fact verification based on struc- tured data is important for many real-life ap- plications and is a challenging research prob- lem, particularly when it involves both sym- bolic operations and informal inference based on language understanding. In this paper, we present a Program-enhanced Verbalization and Graph ATtention Network (ProgVGAT) to integrate programs and execution into tex- tual inference models. Specifically, a verbal- ization with program execution model is pro- posed to accumulate evidences that are embed- ded in operations over the tables. Built on that, we construct the graph attention verification networks, which are designed to fuse differ- ent sources of evidences from verbalized pro- gram execution, program structures, and the original statements and tables, to make the fi- nal verification decision. To support the above framework, we propose a program selection module optimized with a new training strat- egy based on margin loss, to produce more ac- curate programs, which is shown to be effec- tive in enhancing the final verification results. Experimental results show that the proposed framework achieves the new state-of-the-art performance, a 74.4% accuracy, on the bench- mark dataset TABFACT. Our code is avail- able at https://github.com/arielsho/Program- Enhanced-Table-Fact-Checking. 1 Introduction With the overwhelming information available on the Internet, fact verification has become crucial for many applications such as detecting fake news, ru- mors, and political deception (Rashkin et al., 2017; Thorne et al., 2018; Goodrich et al., 2019; Vaibhav et al., 2019; Kry´sci´nski et al., 2019), among others. Existing research has mainly focused on collecting ∗Equal contribution to this work. The work was done during the second author’s visiting to Queen’s University. Year Tournaments Played Avg. Score Scoring Rank 2007 22 72.46 81 2008 29 71.65 22 2009 25 71.90 34 2010 18 73.42 92 2011 11 74.42 125 Table with title ‘Ji-young Oh’ Statement Label Program Ji-young Oh played more tournament in 2008 than any other year. ENTAILED eq { max { all_rows ; tournaments played } ; hop { filter_eq { all_rows ; year ; 2008 } ; tournaments played } } = True Figure 1: An example of fact verification over tables. and processing evidences from unstructured text data (Liu et al., 2020; Nie et al., 2019; Hanselowski et al., 2018; Yoneda et al., 2018), which is only one type of data where important facts exist. Structured and semi-structured data, e.g., tables in relational databases or in the HTML format is also ubiquitous. Performing fact validation based on structured data is important yet challenging and further study is highly desirable. Fig. 1 depicts a simplified ex- ample in …

Content truncated to 500 words