Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4894–4903 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 4894 Explainable Tsetlin Machine Framework for Fake News Detection with Credibility Score Assessment Bimal Bhattarai, Ole-Christoffer Granmo, Lei Jiao University of Agder Norway {bimal.bhattarai, ole.granmo, lei.jiao}@uia.no Abstract The proliferation of fake news, i.e., news intentionally spread for misinformation, poses a threat to individuals and society. De- spite various fact-checking websites such as PolitiFact, robust detection techniques are required to deal with the increase in fake news. Several deep learning models show promising results for fake news classification, however, their black-box nature makes it difficult to explain their classification decisions and quality-assure the models. We here address this problem by proposing a novel interpretable fake news detection framework based on the recently introduced Tsetlin Machine (TM). In brief, we utilize the conjunctive clauses of the TM to capture lexical and semantic properties of both true and fake news text. Further, we use clause ensembles to calculate the credibility of fake news. For evaluation, we conduct experiments on two publicly available datasets, PolitiFact and GossipCop, and demonstrate that the TM framework significantly outperforms previously published baselines by at least 5% in terms of accuracy, with the added benefit of an interpretable logic-based representation. In addition, our approach provides a higher F1-score than BERT and XLNet, however, we obtain slightly lower accuracy. We finally present a case study on our model’s explainability, demonstrating how it decomposes into meaningful words and their negations. Keywords: Fake News Detection, Tsetlin Machine, Human-Interpretable, Language Models, Text Classification, Ex- plainable. 1. Introduction Social media platforms on the Internet have become an integral part of everyday life, and more and more people obtain news from social rather than traditional media, such as newspapers. Key drivers include cost- effectiveness, freedom to comment, and shareability among friends. Despite these advantages, social me- dia exposes people to abundant misinformation. Fake news stories are particularly problematic as they seek to deceive people for political and financial gain (Got- tfried and Shearer, 2016)(Shearer and Mitchell, 2021). In recent years, we have witnessed extensive growth of fake news in social media, spread across news blogs, Twitter, and other social platforms. At present, most online misinformation is manually written (Vargo et al., 2018). However, natural language models like GPT- 3 enable the automatic generation of realistic-looking fake news, which may accelerate future growth. Such growth is problematic as most people nowadays read news stories from social media and news blogs (All- cott and Gentzkow, 2017). Indeed, the spread of fake news poses a severe threat to journalism, individuals, and society. It has the potential of breaking societal belief systems and encourages biases and false hopes. For instance, fake news related to religion or gender inequality can produce harmful prejudices. Fake news can also trigger violence or conflict. When people are frequently exposed to fake news, they tend to distrust real news, affecting their ability to distinguish between truth and …

Content truncated to 500 words