Evaluating adversarial attacks against multiple fact verification systems

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2944–2953, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 2944 Evaluating adversarial attacks against multiple fact verification systems James Thorne University of Cambridge jt719@cam.ac.uk Andreas Vlachos University of Cambridge av308@cam.ac.uk Christos Christodoulopoulos Amazon chrchrs@amazon.co.uk Arpit Mittal Amazon mitarpit@amazon.co.uk Abstract Automated fact verification has been progress- ing owing to advancements in modeling and availability of large datasets. Due to the na- ture of the task, it is critical to understand the vulnerabilities of these systems against adver- sarial instances designed to make them predict incorrectly. We introduce two novel scoring metrics, attack potency and system resilience which take into account the correctness of the adversarial instances, an aspect often ignored in adversarial evaluations. We consider six fact verification systems from the recent Fact Extraction and VERification (FEVER) chal- lenge: the four best-scoring ones and two baselines. We evaluate adversarial instances generated by a recently proposed state-of-the- art method, a paraphrasing method, and rule- based attacks devised for fact verification. We find that our rule-based attacks have higher po- tency, and that while the rankings among the top systems changed, they exhibited higher re- silience than the baselines. 1 Introduction Fact verification is the task of predicting whether claims can be supported or refuted by evidence. Advances in this task have been achieved through improved modelling and the availability of re- sources to train and validate systems (e.g. Wang (2017); Baly et al. (2018b); Thorne et al. (2018)). As this is a task with potentially sensitive appli- cations like propaganda (Baly et al., 2018a) or bi- ased news detection (Potthast et al., 2018), it is critical to understand how systems and models be- have when exposed to real-world data and how de- ficiencies in their training data may contribute to this. It has been observed in related NLP tasks that as models become more complex, it is diffi- cult to fully understand and characterize their be- haviour (Samek et al., 2017). And from an NLP perspective, there has been an ongoing discussion Original REFUTED Instance: Bullitt is a movie directed by Phillip D’Antoni Adversarial REFUTED Instance: There is a movie directed by Phillip D’Antoni called Bullitt. Adversarial SUPPORTED Instance: Bullitt is not a movie directed by Phillip D’Antoni Evidence: Bullitt is a 1968 American action thriller film directed by Peter Yates and produced by Philip D’Antoni Figure 1: Adversarial instances generated through rule- based transformations of existing claims as to what extent these models understand lan- guage (Jia and Liang, 2017) or they are exploit- ing unintentional biases and cues that are present in the datasets they are trained on (Poliak et al., 2018; Gururangan et al., 2018). One of the diagnostic tools for understand- ing how models behave is adversarial evaluation, where data that is deliberately designed to induce classification errors is used to expose “blind spots” of a system. While there are many recently …

Content truncated to 500 words