Learn2Weight: Parameter Adaptation against Similar-domain Adversarial Attacks

Proceedings of the 29th International Conference on Computational Linguistics, pages 4832–4843 October 12–17, 2022. 4832 Learn2Weight: Parameter Adaptation against Similar-domain Adversarial Attacks Siddhartha Datta University of Oxford siddhartha.datta@cs.ox.ac.uk Abstract Recent work in black-box adversarial attacks for NLP systems has attracted much attention. Prior black-box attacks assume that attackers can observe output labels from target models based on selected inputs. In this work, in- spired by adversarial transferability, we pro- pose a new type of black-box NLP adversarial attack that an attacker can choose a similar do- main and transfer the adversarial examples to the target domain and cause poor performance in target model. Based on domain adaptation theory, we then propose a defensive strategy, called Learn2Weight, which trains to predict the weight adjustments for a target model in order to defend against an attack of similar- domain adversarial examples. Using Amazon multi-domain sentiment classification datasets, we empirically show that Learn2Weight is ef- fective against the attack compared to standard black-box defense methods such as adversarial training and defensive distillation. This work contributes to the growing literature on ma- chine learning safety. 1 Introduction As machine learning models are applied to more and more real-world tasks, addressing machine learning safety is becoming an increasingly press- ing issue. Deep learning algorithms have been shown to be vulnerable to adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2014; Pa- pernot et al., 2016a). In particular, prior black-box adversarial attacks assume that the adversary is not aware of the target model architecture, parameters or training data, but is capable of querying the tar- get model with supplied inputs and obtaining the output predictions. The phenomenon that adver- sarial examples generated from one model may also be adversarial to another model is known as adversarial transferability (Szegedy et al., 2013). Motivated by adversarial transferability, we con- jecture another black-box attack pipeline where the adversary does not even need to have access to the target model nor query labels from crafted inputs. Instead, as long as the adversary knows the task of the target, they can choose a similar domain to build a substitute model, and then attack the target model with adversarial examples that are generated from the attack domain. The similar-domain adversarial attack may be more practical than prior blackbox attacks as label querying from the target model is not needed. This attack can be illustrated with the following example (Figure 1b) in medical insurance fraud (Finlayson et al., 2019). Insurance companies may use hypo- thetical opioid risk models to classify the likelihood (high/low) of a patient to abuse the opioids to be prescribed, based on the patient’s medical history as text input. Physicians can run the original pa- tient history through the attack pipeline to generate an adversarial patient history, where the original is more likely to be rejected ("High" risk) and the adversarial is more likely to be accepted ("Low" risk). Perturbations in patient history could be, for example, a slight perturbation from "alcohol abuse" to "alcohol dependence", and it may …

Content truncated to 500 words