FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 10421–10440 July 9-14, 2023 ©2023 Association for Computational Linguistics FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering Anku Rani1 S.M Towhidul Islam Tonmoy2 Dwip Dalal3 Shreya Gautam4 Megha Chakraborty1 Aman Chadha†5,6 Amit Sheth1 Amitava Das1 1University of South Carolina, USA 2IUT, Bangladesh 3IIT Gandhinagar, India 4BIT Mesra, India 5Stanford University, USA 6Amazon AI, USA arani@mailbox.sc.edu amitava@mailbox.sc.edu Abstract Automatic fact verification has received signif- icant attention recently. Contemporary auto- matic fact-checking systems focus on estimat- ing truthfulness using numerical scores which are not human-interpretable. A human fact- checker generally follows several logical steps to verify a verisimilitude claim and conclude whether it’s truthful or a mere masquerade. Popular fact-checking websites follow a com- mon structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspect- based (delineating which part(s) are true and which are false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final ver- dict. In this paper, we propose a 5W frame- work (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we present a semi-automatically gen- erated dataset called FACTIFY-5WQA, which consists of 391, 041 facts along with relevant 5W QAs – underscoring our major contribu- tion to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those an- swers from evidence documents, which can serve as a baseline for future research in the field. Lastly, we propose a robust fact veri- fication system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https: //github.com/ankuranii/acl-5W-QA 1 Fact checking demands aspect-based explainability Manual fact-checking is a time-consuming task. To assess the truthfulness of a claim, a journalist would either need to search online, offline, or both, brows- †Work does not relate to the position at Amazon. ing through a multitude of sources while also ac- counting for the perceived reliability of each source. The final verdict can then be obtained via assimi- lation and/or comparison of the facts derived from said sources. This process can take professional fact-checkers several hours or days (Hassan et al., 2019) (Adair et al., 2017), depending on the inher- ent complexity of the claim. There are several contemporary practices that journalists use for the manual verification of a claim. These methods can be categorized into four broad categories (Posetti et al., 2018): 1. Research and fact-checking: This involves carefully researching the claim and verify- ing its accuracy using reliable and credible sources such as news services, academic stud- ies, and government data. 2. Interviews and expert opinions: This in- volves speaking with experts in the relevant field and asking for their opinions …

Content truncated to 500 words