Unsupervised Fact Checking by Counter-Weighted Positive and Negative Evidential Paths in A Knowledge Graph

Proceedings of the 28th International Conference on Computational Linguistics, pages 1677–1686 Barcelona, Spain (Online), December 8-13, 2020 1677 Unsupervised Fact Checking by Counter-Weighted Positive and Negative Evidential Paths in A Knowledge Graph Jiseong Kim Semantic Web Research Center School of Computing KAIST jiseong@kaist.ac.kr Key-Sun Choi Semantic Web Research Center School of Computing KAIST kschoi@kaist.ac.kr Abstract Misinformation spreads across media, community, and knowledge graphs in the Web by not only human agents but also information extraction algorithms that extract factual statements from unstructured textual data to populate the existing knowledge graphs. Traditional fact checking by experts or crowds is increasingly difficult to keep pace with the volume of newly created misinformation in the Web. Therefore, it is important and necessary to enhance the com- putational ability to determine whether a given factual statement is truthful or not. We view this problem as a truth scoring task in a knowledge graph. We present a novel rule-based approach that finds positive and negative evidential paths in a knowledge graph for a given factual state- ment, and calculates a truth score for the given statement by unsupervised ensemble of the found positive and negative evidential paths. For example, we can determine the factual statement “United States is the birth place of Barack Obama” as truthful if there is the positive evidential path (Barack Obama, birthPlace, Hawaii) ∧ (Hawaii, country, United States) in a knowledge graph. For another example, we can determine the factual statement “Canada is the nationality of Barack Obama” as untruthful if there is the negative evidential path (Barack Obama, nationality, United States) ∧ (United States, ≠, Canada) in a knowledge graph. For evaluating on a real-world situation, we constructed an evaluation dataset by labeling truth or untruth label on factual statements that were extracted from Wikipedia texts by using the state-of-the-art BERT-based information extraction system. Our evaluation results show that our approach outperforms the state-of-the-art unsupervised approaches significantly by up to 0.12 AUC-ROC and even outperforms the supervised approach by up to 0.05 AUC- ROC not only in our dataset but also in the two different standard datasets. 1 Introduction1 Misinformation in the Web creates a situation in which false statements compete for attention to true statement necessary for users and applications. Misinformation in media and community makes difficult for users to search the information they need and misinformation in knowledge graphs makes difficult for applications to get the outputs they expect. This problem is common and getting worse in modern digital society. Although a lot of information in the Web is a good resource, there is certainly no guar- antee that a given factual statement is truth or not. In order not to be fooled by false statements, it is necessary to separate truth from untruth by assessing truthfulness of factual statements. We represent a factual statement as a triple (subject, predicate, object) where subject and object are entities that have a relationship between them as indicated by predicate. For ex- ample, “Leonardo da Vinci is known for Mona Lisa” …

Content truncated to 500 words