A Zero-Shot Claim Detection Framework Using Question Answering

Proceedings of the 29th International Conference on Computational Linguistics, pages 6927–6933 October 12–17, 2022. 6927 A Zero-Shot Claim Detection Framework using Question Answering Revanth Gangi Reddy1, Sai Chetan1, Yi R. Fung1, Kevin Small2, Heng Ji1 1University of Illinois at Urbana-Champaign 2Amazon Alexa AI {revanth3, scc8, yifung2, hengji}@illinois.edu smakevin@amazon.com Abstract In recent years, there has been an increasing interest in claim detection as an important build- ing block for misinformation detection. This involves detecting more fine-grained attributes relating to the claim, such as the claimer, claim topic, claim object pertaining to the topic, etc. Yet, a notable bottleneck of existing claim de- tection approaches is their portability to emerg- ing events and low-resource training data set- tings. In this regard, we propose a fine-grained claim detection framework that leverages zero- shot Question Answering (QA) using directed questions to solve a diverse set of sub-tasks such as topic filtering, claim object detection, and claimer detection. We show that our ap- proach1 significantly outperforms various zero- shot, few-shot and task-specific baselines on the NEWSCLAIMS benchmark (Reddy et al., 2021). 1 Introduction Claim detection over news involves identifying claims related to various topics in news articles. Identifying such claims is a crucial first step for fighting misinformation and disinformation online. However, such harmful content can evolve rapidly, triggered by relatively new events which can gain extensive media coverage within a short time span. Hence, claim detection in such scenarios requires systems that are able to adapt quickly, by working well under zero-shot or few-shot settings. Towards this goal, Reddy et al. (2021) propose a new benchmark, NEWSCLAIMS, that evaluates claim detection for previously unseen topics with- out access to any training data. Given a collec- tion of news articles, the task involves identify- ing claims that are related to a pre-defined set of topics, along with extracting attributes for each claim, such as claim span, claim object, claimer 1Code is available here: https://github.com/blender- nlp/NewsClaims/tree/main/zero-shot-qa Figure 1: An overview of our claim detection frame- work. We leverage zero-shot QA for claim topic filter- ing, claim object detection and claimer detection for the claims spotted by ClaimBuster (Hassan et al., 2017). and stance. Essentially, this benchmark extends the claim detection task to involve extracting additional background attributes relating to the claim, such as the claim object and claimer. Furthermore, the claimer detection sub-task within NEWSCLAIMS requires considerable document-level reasoning, making it harder than existing attribution tasks (Pareti, 2016; Newell et al., 2018), which mainly in- volve sentence-level reasoning. Reddy et al. (2021) propose various baselines for each sub-task in the NEWSCLAIMS benchmark, which involve zero- shot and few-shot approaches in addition to base- lines trained using task-specific data. To handle this scenario in a low-resource setting, we hypothesize that identifying claim topics and extracting corresponding claim attributes can be formulated as a question answering task. Hence, we propose to leverage a Question Answering (QA) system for the claim detection task and show that the same QA model can be used to solve multiple sub-tasks within claim …

Content truncated to 500 words