A Multi-Level Attention Model for Evidence-Based Fact Checking

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2447–2460 August 1–6, 2021. ©2021 Association for Computational Linguistics 2447 A Multi-Level Attention Model for Evidence-Based Fact Checking Canasai Kruengkrai Junichi Yamagishi National Institute of Informatics, JapanXin Wang {canasai,jyamagishi,wangxin}@nii.ac.jp Abstract Evidence-based fact checking aims to verify the truthfulness of a claim against evidence ex- tracted from textual sources. Learning a rep- resentation that effectively captures relations between a claim and evidence can be challeng- ing. Recent state-of-the-art approaches have developed increasingly sophisticated models based on graph structures. We present a simple model that can be trained on sequence struc- tures. Our model enables inter-sentence atten- tions at different levels and can benefit from joint training. Results on a large-scale dataset for Fact Extraction and VERification (FEVER) show that our model outperforms the graph- based approaches and yields 1.09% and 1.42% improvements in label accuracy and FEVER score, respectively, over the best published model.1 1 Introduction False or misleading claims spread through online media faster and wider than the truth (Vosoughi et al., 2018). False claims can occur in many differ- ent forms, e.g., fake news, rumors, hoaxes, propa- ganda, etc. Identifying false claims that are likely to cause harm in the real world is important. Gen- erally, claims can be categorized into two types: verifiable and unverifiable. Verifiable claims can be confirmed to be true or false as guided by evidence from credible sources, while unverifiable claims cannot be confirmed due to insufficient informa- tion. Verifying the truthfulness of a claim with respect to evidence can be regarded as a special case of rec- ognizing textual entailment (RTE) (Dagan et al., 2006) or natural language inference (NLI) (Bow- man et al., 2015), where the premise (evidence) is not given. Thus, the task of claim verification is to 1The code and model checkpoints are available at: https://github.com/nii-yamagishilab/mla. 8143 Moscovium is a transactinide element. SUPPORTED [Moscovium] Moscovium is a superheavy synthetic element with symbol Mc and atomic number 115.0 In the periodic table, it is a p-block transactinide element.7 [Transactinide_element] In chemistry, transactinide elements (also, transactinides, or super-heavy elements) are the chemical elements with atomic numbers from 104 to 120.0 ID: Claim: Label: Evidence: 201459 A dynamic web page does not involve computer programming. REFUTED [Web_page] A static web page is delivered exactly as stored, as web content in the web server's file system, while a dynamic web page is generated by a web application that is driven by server-side software or client-side scripting.14 [Dynamic_web_page] A dynamic web page is then reloaded by the user or by a computer program to change some variable content.9 ID: Claim: Label: Evidence: Figure 1: Examples from the FEVER dev set, where true evidence sentences are present in the selected sen- tences, and veracity relation labels are correctly pre- dicted by our proposed model. Wikipedia article titles are in [italics]. Superscripts indicate the positions of the sentences in each article. first retrieve documents relevant to a given claim from textual sources, then select sentences likely to …

Content truncated to 500 words