Validation of Facts Against Textual Sources

Proceedings of Recent Advances in Natural Language Processing, pages 895–903, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_104 895 Validation of Facts Against Textual Sources Vamsi Krishna Pendyala Simran sinha Satya Prakash Shriya Reddy Anupam Jamatia Department of Computer Science and Engineering National Institute of Technology Agartala, Tripura, India {krishnapendiala, simsinha32}@gmail.com {satyaprakash30497, shriyavipul90, anupamjamatia}@gmail.com Abstract In today’s world, the spreading of fake news has become facile through social me- dia which diffuses rapidly and can be be- lieved easily. Fact Checkers or Fact Veri- fiers are the need of the hour. In this paper, we propose a system which would verify a claim(fact) against a textual source pro- vided and classify the claim to be true, false, out-of-context or inappropriate with respect to that source. This would help us to verify a fact as well as know about the source of our knowledge base against which the fact is being verified. We used a two-step approach to achieve our goal. First step is about retrieving the ev- idence related to the claims from the tex- tual source. Next step is the classification of the claim as true, false, inappropriate and out of context with respect to the ev- idence using a modified version of textual entailment module. The accuracy of the best performing system is 64.95%. 1 Introduction Fact Checking is one of the biggest buzzwords of this era. Many a time, the news transmitted through social media can be moulded and altered when so many people share information and it is hard to discern between fact and fiction. So there is a need to verify every piece of infor- mation we observe in our day-to-day life to be true or false. A solution to this problem is, we check each claim or fact manually against a reli- able source and then label the claim or fact to be either true or false which is time consuming for large data. Vlachos and Riedel (2014) discussed the fact verification process as an ordinal text clas- sification task, where they created a data-set using the manually annotated data present on sites like PolitiFact, FactCheck, FullFact. The FakeNews Challenge1 by Riedel et al. (2017) addresses fact- checking as a simple instance detection problem, which mainly checks whether the given instance is in accordance with the article or not. Recently Thorne et al. (2018a) published a huge data set to deeply understand the process of large scale fact- checking. All of the above approaches rely com- pletely upon the source against which data is to be verified, but in some cases, the source text might contain limited or no amount of information about the claim and a claim might be Out-of-Context of this source text. This leads to a problem of clas- sifying a claim to be within the scope/context of the source text or not. Concretely, we do need a system which would not only classify a claim to be true or false but also check whether the source text is sufficient enough to classify …

Content truncated to 500 words