Varifocal Question Generation for Fact-checking

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2532 - 2544 December 7-11, 2022 ©2022 Association for Computational Linguistics Varifocal Question Generation for Fact-checking Nedjma Ousidhoum∗ Zhangdie Yuan∗ Andreas Vlachos Department of Computer Science and Technology University of Cambridge ndo24,zy317,av308@cam.ac.uk Abstract Fact-checking requires retrieving evidence re- lated to a claim under investigation. The task can be formulated as question generation based on a claim, followed by question answer- ing. However, recent question generation ap- proaches assume that the answer is known and typically contained in a passage given as in- put, whereas such passages are what is being sought when verifying a claim. In this paper, we present Varifocal, a method that generates questions based on different focal points within a given claim, i.e. different spans of the claim and its metadata, such as its source and date. Our method outperforms previous work on a fact-checking question generation dataset on a wide range of automatic evaluation metrics. These results are corroborated by our manual evaluation, which indicates that our method generates more relevant and informative ques- tions. We further demonstrate the potential of focal points in generating sets of clarification questions for product descriptions. 1 Introduction The growing amount of information online and its impact have increased the need for fact-checking, i.e. judging whether a claim is true or false. To determine the truthfulness of a claim, fact-checkers need to answer questions related to the claim, world knowledge within its time frame, local politics, etc. (Graves, 2017). Using questions and answers has also been shown to be an effective way of conveying fact-checks. For instance, Altay et al. (2021) found that presenting information related to COVID-19 as answers to questions improved attitudes towards vaccination more than merely pre- senting the relevant facts. As professional fact-checkers can spend a day to verify a single claim depending on its complex- ity (Adair et al., 2017; Hassan et al., 2017), there *Equal contribution. has been a growing focus on how to accelerate the fact-checking process via automation (Cohen et al., 2011; Graves and Anderson, 2020). Fan et al. (2020) showed that generating questions and an- swering them reduces the time spent on verification by approximately 20%. Fact verification questions tackle information that is missing from the claim, which renders the generation task challenging yet useful for assisting professionals. Previous work on question generation assumes that the answer is known, typically contained in a passage given in the input (Rajpurkar et al., 2016; Duan et al., 2017; Wang et al., 2017). Such pas- sages, though, are what is being sought when fact- checking a claim. The only exception is recent work on clarification questions (Rao and Daumé, 2019; Majumder et al., 2021). However, work in this area examines a specific narrow domain where a limited number of questions can be asked, e.g. questions related to product descriptions in the Amazon dataset of McAuley and Yang (2016), or dialogues in the Ubuntu dataset (Lowe et al., 2015), which is not …

Content truncated to 500 words