Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7701 - 7715 December 7-11, 2022 ©2022 Association for Computational Linguistics Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter Megha Sundriyal1∗, Atharva Kulkarni1∗, Vaibhav Pulastya1, Md Shad Akhtar1, Tanmoy Chakraborty2 1IIIT Delhi, India, 2IIT Delhi, India {meghas, atharvak, vaibhav17271, shad.akhtar}@iiitd.ac.in, tanchak@ee.iitd.ac.in Abstract The widespread diffusion of medical and po- litical claims in the wake of COVID-19 has led to a voluminous rise in misinformation and fake news. The current vogue is to employ manual fact-checkers to efficiently classify and verify such data to combat this avalanche of claim-ridden misinformation. However, the rate of information dissemination is such that it vastly outpaces the fact-checkers’ strength. Therefore, to aid manual fact-checkers in elim- inating the superfluous content, it becomes im- perative to automatically identify and extract the snippets of claim-worthy (mis)information present in a post. In this work, we introduce the novel task of Claim Span Identification (CSI). We propose CURT, a large-scale Twitter cor- pus with token-level claim spans on more than 7.5k tweets. Furthermore, along with the stan- dard token classification baselines, we bench- mark our dataset with DABERTa, an adapter- based variation of RoBERTa. The experimental results attest that DABERTa outperforms the baseline systems across several evaluation met- rics, improving by about 1.5 points. We also report detailed error analysis to validate the model’s performance along with the ablation studies. Lastly, we release our comprehensive span annotation guidelines for public use. 1 Introduction The swift acceleration of Online Social Media (OSM) platforms has led to tremendous democ- ratized content creation and information exchange. Consequently, these platforms serve as ideal breed- ing grounds for malicious rumormongers and tale- bearers, abetting a colossal upsurge of misinfor- mation. Such misinformation manifests in many ways, including bogus claims, fabricated informa- tion, and rumors. The massive COVID-19 ‘In- fodemic’ (Naeem and Bhatti, 2020) is one such malignant byproduct that led to the rampant spread ∗Equal contribution RT @PirateAtLaw: No no no. Corona beer is the cure not the disease. We don’t have evidence but we are positive our wine keeps you from getting #COVID19 if you drink enough of it. Better alternative to #DisinfectantInjection don’t you think? #winecures. RT @angeliicamdc: Mexicans are immune to the coronavirus because we have sana sana colita de rana @adamseconomics Vaccine is probably made from Chinese ingredients sourced in Wuhan. Figure 1: Examples of claim tweets and their ground truth claim spans highlighted in boldface text (blue). of political and social calumny (Ferrara, 2020; Mar- golin, 2020; Ziems et al., 2020), accompanied by counterfeit pharmaceutical claims (O’Connor and Murphy, 2020). Therefore, finding such claim- ridden posts on OSM platforms, investigating their plausibility, and differentiating the credible claims from the apocryphal ones has risen to be a pertinent research problem in Argument Mining (AM). ‘Claim’, as coined by Toulmin (2003), is ‘an assertion that deserves our attention’. It is the key component of any argument (Daxenberger et al., 2017). Consider the second tweet, ‘We don’t …

Content truncated to 500 words