Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1496 - 1511 July 10-15, 2022 ©2022 Association for Computational Linguistics Natural Language Inference with Self-Attention for Veracity Assessment of Pandemic Claims Miguel Arana-Catania1,2, Elena Kochkina3,2, Arkaitz Zubiaga3, Maria Liakata3,2, Rob Procter1,2, Yulan He1,2 1Department of Computer Science, University of Warwick, UK 2The Alan Turing Institute, UK 3Queen-Mary University of London, UK Abstract We present a comprehensive work on auto- mated veracity assessment from dataset cre- ation to developing novel methods based on Natural Language Inference (NLI), focusing on misinformation related to the COVID-19 pan- demic. We first describe the construction of the novel PANACEA dataset consisting of hetero- geneous claims on COVID-19 and their respec- tive information sources. The dataset construc- tion includes work on retrieval techniques and similarity measurements to ensure a unique set of claims. We then propose novel techniques for automated veracity assessment based on Natural Language Inference including graph convolutional networks and attention based ap- proaches. We have carried out experiments on evidence retrieval and veracity assessment on the dataset using the proposed techniques and found them competitive with SOTA methods, and provided a detailed discussion. 1 Introduction In recent years, and particularly with the emer- gence of the COVID-19 pandemic, significant ef- forts have been made to detect misinformation on- line with the aim of mitigating its impact. With this objective, researchers have proposed numerous approaches and released datasets that can help with the advancement of research in this direction. Most existing datasets (D’Ulizia et al., 2021) fo- cus on a single medium (e.g., Twitter, Facebook, or specific websites), a unique information domain (e.g., health information, general news, or scholarly papers), a type of information (e.g., general claims or news), or a specific application (e.g., verifying claims, or retrieving useful information). This in- evitably results in a limited focus on what is a com- plex, multi-faceted phenomenon. With the aim of furthering research in this direction, the contribu- tions of our work are twofold: (1) creating a new comprehensive dataset of misinformation claims, and (2) introducing two novel approaches to verac- ity assessment. In the first part of our work, we contribute to the global effort on addressing misinformation in the context of COVID-19 by creating a dataset for PANdemic Ai Claim vEracity Assessment, called the PANACEA dataset. It is a new dataset that combines different data sources with different foci, thus enabling a comprehensive approach that com- bines different media, domains and information types. To this effect our dataset brings together a heterogeneous set of True and False COVID claims and online sources of information for each claim. The collected claims have been obtained from on- line fact-checking sources, existing datasets and re- search challenges. We have identified a large over- lap of claims between different sources and even within each source or dataset. Thus, given the chal- lenges of aggregating multiple data sources, much of our efforts in dataset construction has focused …

Content truncated to 500 words