Exploring Unsupervised Semantic Similarity Methods for Claim Verification in Health Care News Articles

Proceedings of Recent Advances in Natural Language Processing, pages 440–447 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_049 440 Exploring Unsupervised Semantic Similarity Methods for Claim Verification in Health Care News Articles Vishwani Gupta1, Astrid Viciano2, Holger Wormer3, and Najmehsadat Mousavinezhad1 1Fraunhofer IAIS, Sankt Augustin, Germany 2Medien-Doktor Gesundheit, Institut f¨ur Journalistik, TU Dortmund, Dortmund, Germany 3Lehrstuhl f¨ur Wissenschaftsjournalismus, Institut f¨ur Journalistik, TU Dortmund, Dortmund, Germany {vishwani.gupta,Najmehsadat.Mousavinezhad}@iais.fraunhofer.de astrid.viciano@tu-dortmund.de holger.wormer@uni-dortmund.de Abstract In the 21st century, the proliferation of fake information has emerged as a significant threat to society. Particularly, healthcare medical re- porters face challenges when verifying claims related to treatment effects, side effects, and risks mentioned in news articles, relying on scientific publications for accuracy. The accu- rate communication of scientific information in news articles has long been a crucial concern in the scientific community, as the dissemination of misinformation can have dire consequences in the healthcare domain. This paper delves into the application of unsupervised semantic similarity models to facilitate claim verifica- tion for medical reporters, thereby expediting the process. We explore unsupervised multi- lingual evidence retrieval techniques aimed at reducing the time required to obtain evidence from scientific studies. Instead of employing content classification, we propose an approach that retrieves relevant evidence from scientific publications for claim verification within the healthcare domain. Given a claim and a set of scientific publications, our system generates a list of the most similar paragraphs containing supporting evidence. Furthermore, we evalu- ate the performance of state-of-the-art unsuper- vised semantic similarity methods in this task. As the claim and evidence are present in a cross- lingual space, we find that the XML-RoBERTa model exhibits high accuracy in achieving our objective. 1 Introduction The rise of misinformation has been greatly ampli- fied by the advent of social media, primarily due to its increased dissemination and influence. One prominent manifestation of this issue is vaccine hesitancy, which has had significant societal reper- cussions. To illustrate this, a web-based survey (Neely et al., 2022) was conducted in June 2021 among 600 adults in Florida, revealing substan- tial exposure to COVID-19 vaccine misinformation among participants. Approximately 73% reported encountering misinformation in the past six months. An overview of current fake news research is given by (Kim et al., 2021). Through the convergence of computational and social science research, they delve into the significance and trajectory of enhanc- ing ”digital media literacy” in diverse contexts of news generation and consumption. Detecting misinformation has emerged as a crit- ical challenge due to the rapid dissemination of news and the potentially severe consequences asso- ciated with false information. However, only a lim- ited number of approaches have been developed to address the dynamic, versatile, and fast-spreading nature of fake news editorials. This challenge be- comes even more pronounced in the healthcare domain, where the availability of training data is scarce, and pre-trained models may not be read- ily applicable. While supervised models rely on manually annotated training data, an unsupervised evidence retrieval and verification approach proves more suitable for quick response and works effec- …

Content truncated to 500 words