Generating Label Cohesive and Well-Formed Adversarial Claims

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3168–3177, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3168 Generating Label Cohesive and Well-Formed Adversarial Claims Pepa Atanasova∗ and Dustin Wright∗ and Isabelle Augenstein Dept. of Computer Science University of Copenhagen Denmark {pepa|dw|augenstein}@di.ku.dk Abstract Adversarial attacks reveal important vulnera- bilities and flaws of trained models. One po- tent type of attack are universal adversarial triggers, which are individual n-grams that, when appended to instances of a class under attack, can trick a model into predicting a tar- get class. However, for inference tasks such as fact checking, these triggers often inadver- tently invert the meaning of instances they are inserted in. In addition, such attacks produce semantically nonsensical inputs, as they sim- ply concatenate triggers to existing samples. Here, we investigate how to generate adversar- ial attacks against fact checking systems that preserve the ground truth meaning and are se- mantically valid. We extend the HotFlip attack algorithm used for universal trigger genera- tion by jointly minimizing the target class loss of a fact checking model and the entailment class loss of an auxiliary natural language in- ference model. We then train a conditional language model to generate semantically valid statements, which include the found universal triggers. We find that the generated attacks maintain the directionality and semantic valid- ity of the claim better than previous work. 1 Introduction Adversarial examples (Goodfellow et al., 2015; Szegedy et al., 2013) are deceptive model inputs designed to mislead an ML system into making the wrong prediction. They expose regions of the input space that are outside the training data distri- bution where the model is unstable. It is important to reveal such vulnerabilities and correct for them, especially for tasks such as fact checking (FC). In this paper, we explore the vulnerabilities of FC models trained on the FEVER dataset (Thorne et al., 2018), where the inference between a claim and evidence text is predicted. We particularly ∗denotes equal contribution Dissociative�disorders have�been�attributed�to disruptions�in�memory caused�by�trauma�or�other forms�of�stress. SUPPORTS REFUTES Dissociative�identity disorder,�or�DID,�may�be the�result�of�memory disruptions�that�have�been induced�by�psychological trauma. CLAIM don,already,more,during,home GPT-2 Claim Generation Dissociative�disorders�have�been�attributed�to�disrupted brain�activity�during�trauma�or�other�forms�of�stress. TRIGGERS Trigger Generation EVIDENCE Figure 1: High level overview of our method. First, universal triggers are discovered for flipping a source to a target label (e.g. SUPPORTS → REFUTES). These triggers are then used to condition the GPT-2 language model to generate novel claims with the original label, including at least one of the found triggers. construct universal adversarial triggers (Wallace et al., 2019) – single n-grams appended to the input text that can shift the prediction of a model from a source class to a target one. Such adversarial examples are of particular concern, as they can apply to a large number of input instances. However, we find that the triggers also change the meaning of the claim such that the true label is in fact the target class. For example, when at- tacking a claim-evidence pair with a ‘SUPPORTS’ label, a common unigram found to be …

Content truncated to 500 words