What is Your Article Based On? Inferring Fine-grained Provenance

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 5894–5903 August 1–6, 2021. ©2021 Association for Computational Linguistics 5894 What is Your Article Based On? Inferring Fine-grained Provenance Yi Zhang, Zachary G. Ives, Dan Roth Department of Computer and Information Science University of Pennsylvania, Philadelphia, PA, USA {yizhang5, zives, danroth}@cis.upenn.edu Abstract When evaluating an article and the claims it makes, a critical reader must be able to as- sess where the information presented comes from, and whether the various claims are mu- tually consistent and support the conclusion. This motivates the study of claim provenance, which seeks to trace and explain the origins of claims. In this paper, we introduce new tech- niques to model and reason about the prove- nance of multiple interacting claims, including how to capture fine-grained information about the context. Our solution hinges on first identi- fying the sentences that potentially contain im- portant external information. We then develop a query generator with our novel rank-aware cross attention mechanism, which aims at gen- erating metadata for the source article, based on the context and signals collected from a search engine. This establishes relevant search queries, and it allows us to obtain source arti- cle candidates for each identified sentence and propose an ILP based algorithm to infer the best sources. We experiment with a newly cre- ated evaluation dataset 1, Politi-Prov, based on fact-checking articles from www.politifa ct.com; our experimental results show that our solution leads to a significant improvement over baselines. 1 Introduction Misinformation is on the rise, and people are fight- ing it with fact checking. However, most of the work in the current literature (Thorne et al., 2018; Zhang et al., 2019; Barr´on-Cedeno et al., 2020; Hidey et al., 2020) focuses on automating fact- checking for a single claim. In reality, a claim can be complex, and proposed as a conclusion of an article. Therefore, understanding what infor- mation supports the article, especially information 1The data and the code will be available at http://co gcomp.org/page/publication view/944 Figure 1: An example of a claim (in the red box) with its article. Sentence 1 and sentence 2 (blue boxes) show examples from the article. Each sentence refers to ex- ternal information: source article 1 and 2, respectively, with accompanying urls. that was not originated within the same article, and where it originates from, are very important for readers who want to determine whether they can believe the claim. Figure 1 shows an example of such a claim, “Marco Rubio says Anthony Fauci lies about masks. Fauci didn’t.”2 with its article from politifact.com. A critical reader of the con- tent will find that several major sources support the author’s claim: Source article 1 in the figure is CBS News,“60 Minutes” interview with Anthony Fauci, on March 8, 2020, which reveals that Dr. Fauci’s main point was to preserve masks for those who were already ill and people providing care. If readers can …

Content truncated to 500 words