A Re-evaluation of Knowledge Graph Completion Methods

37478

Authors

nan

Abstract

AbstractKnowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs. A vast number of state-of-the-art KGC techniques have got published at top conferences in several research fields, including data mining, machine learning, and natural language processing. However, we notice that several recent papers report very high performance, which largely outperforms previous state-of-the-art methods. In this paper, we find that this can be attributed to the inappropriate evaluation protocol used by them and propose a simple evaluation protocol to address this problem. The proposed protocol is robust to handle bias in the model, which can substantially affect the final results. We conduct extensive experiments and report performance of several existing methods using our protocol. The reproducible code has been made publicly available.

Keywords

nan

Full Content

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 505–514 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 505 GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media Yi-Ju Lu Department of Statistics National Cheng Kung University Tainan, Taiwan l852888@gmail.com Cheng-Te Li Institute of Data Science National Cheng Kung University Tainan, Taiwan chengte@mail.ncku.edu.tw Abstract This paper solves the fake news detection prob- lem under a more realistic scenario on so- cial media. Given the source short-text tweet and the corresponding sequence of retweet users without text comments, we aim at pre- dicting whether the source tweet is fake or not, and generating explanation by highlight- ing the evidences on suspicious retweeters and the words they concern. We develop a novel neural network-based model, Graph-aware Co- Attention Networks (GCAN), to achieve the goal. Extensive experiments conducted on real tweet datasets exhibit that GCAN can signifi- cantly outperform state-of-the-art methods by 16% in accuracy on average. In addition, the case studies also show that GCAN can produce reasonable explanations. 1 Introduction Social media is indispensable in people’s daily life, where users can express themselves, access news, and interact with each other. Information can fur- ther spread through the social network. Opinions and sentiments on source stories can be reflected by user participation and interaction. The conve- nient and low-cost essence of social networking brings collective intelligence, but at the same time leads to a negative by-product, the propagation of misinformation such as fake news. Fake news is a kind of news story possess- ing intentionally false information on social me- dia (Rashkin et al., 2017; Allcott and Gentzkow, 2017). The widespread of fake news can mislead the public, and produce unjust political, economic, or psychological profit for some parties (Horne and Adali, 2017; Allcott and Gentzkow, 2017). Data mining and machine learning techniques were uti- lized to detect fake news (Shu et al., 2017; Cha et al., 2020). Typical approaches rely on the con- tent of new articles to extract textual features, such as n-gram and bag of words, and apply supervised learning (e.g., random forest and support vector ma- chine) for binary classification (Shu et al., 2017). NLP researchers also learn advanced linguistic fea- tures, such as factive/assertive verbs and subjec- tivity (Popat, 2017) and writing styles and consis- tency (Potthast et al., 2018). Multi-modal context information is also investigated, such as user pro- files (Yang et al., 2012; Liu and Wu, 2018) and retweet propagation (Ruchansky et al., 2017; Shu et al., 2019a). Nevertheless, there are still critical challenges in detecting fake news online. First, existing content- based approaches (Castillo et al., 2011; Potthast et al., 2018; Shu et al., 2019a) require documents to be long text, e.g., news articles, so that the rep- resentation of words and sentences can be better learned. However, tweets on social media are usu- ally short text (Yan et al., 2015), which produces severe data sparsity problem. Second, some state- of-the-art models (Ruchansky et al., …

Content truncated to 500 words

Labeled Text

AbstractKnowledge Graph Completion (KGC) aims at automatically predicting missing links for large-scale knowledge graphs. A vast number of state-of-the-art KGC techniques have got published at top conferences in several research fields, including data mining, machine learning, and natural language processing. However, we notice that several recent papers report very high performance, which largely outperforms previous state-of-the-art methods. In this paper, we find that this can be attributed to the inappropriate evaluation protocol used by them and propose a simple evaluation protocol to address this problem. The proposed protocol is robust to handle bias in the model, which can substantially affect the final results. We conduct extensive experiments and report performance of several existing methods using our protocol. The reproducible code has been made publicly available.

Publication Details

Paper ID:
37478
Year:
2020
Publication:
TIPSTER
Publication Type:
conf
DOI:
nan