AttentiveChecker: A Bi-Directional Attention Flow Mechanism for Fact Verification

Proceedings of NAACL-HLT 2019, pages 2218–2222 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2218 AttentiveChecker: A Bi-Directional Attention Flow Mechanism for Fact Verification T.Y.S.S.Santosh, Vishal G, Avirup Saha, Niloy Ganguly Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India {santoshtyss, arsrivish2, saha.avirup}@gmail.com , niloy@cse.iitkgp.ac.in Abstract The recently released FEVER dataset pro- vided benchmark results on a fact-checking task in which given a factual claim, the sys- tem must extract textual evidence (sets of sen- tences from Wikipedia pages) that support or refute the claim. In this paper, we present a completely task-agnostic pipelined system, AttentiveChecker, consisting of three homoge- neous Bi-Directional Attention Flow (BIDAF) networks, which are multi-layer hierarchical networks that represent the context at different levels of granularity. We are the first to ap- ply to this task a bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization. AttentiveChecker can be used to perform doc- ument retrieval, sentence selection, and claim verification. Experiments on the FEVER dataset indicate that AttentiveChecker is able to achieve the state-of-the-art results on the FEVER test set. 1 Introduction The rising influence of fake news poses a clear threat to ethical journalism and the future of democracy. In order to tackle the sheer volume of fake news produced, robust automatic tech- niques to counter it need to be developed. To that end, in order to facilitate researchers to develop algorithms, a number of fact checking datasets have been released in the recent past (Vlachos and Riedel, 2014), (Wang, 2017), (Ferreira and Vla- chos, 2016), (P´erez-Rosas et al., 2017), the 2017 Fake News Challenge (Pomerleau and Rao, 2017) dataset, the dataset released against Triple Scor- ing Task at the WSDM Cup 2017 (Heindorf et al., 2017) etc. However, none of these datasets provide manual annotation for sentence or phrase-level evidence. In this work, we experiment on the Fact Extrac- tion and VERification (FEVER) dataset (Thorne et al., 2018a), which is one of the first which provides sentence-level annotations. (An Arabic corpus (Baly et al., 2018) has been recently re- leased.) A shared task corresponding to the dataset was floated that required verification of an input claim with potential evidence in a large database of about 5 million Wikipedia documents, and also provided a standardized benchmark setting which enabled easy and fair comparison. Table 1 lists the dataset splits and sizes. Split Supports Refutes NotEnough Info train 80,035 29,775 35,639 dev 3,333 3,333 3,333 test 3,333 3,333 3,333 reserved 6,666 6,666 6,666 Table 1: Statistics of claims in FEVER Several attempts have been made to tackle the task defined by FEVER, the most notable ones being (Nie et al., 2018; Yoneda et al., 2018; Hanselowski et al., 2018) which secured the 1st, 2nd and 3rd place on this shared task respectively (Thorne et al., 2018b). Diverse methods were ap- plied, mostly using task-specific features which al- lowed them to beat the baseline given by (Thorne et al., 2018a). In this paper, we propose …

Content truncated to 500 words