TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11975 - 11992 December 7-11, 2022 ©2022 Association for Computational Linguistics TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack Yu Cao1∗, Dianqi Li2, Meng Fang3, Tianyi Zhou4, Jun Gao5, Yibing Zhan6, Dacheng Tao1 1School of Computer Science, The University of Sydney 2University of Washington 3University of Liverpool 4University of Maryland 5Harbin Institute of Technology, Shenzhen 6JD Explore Academy ycao8647@uni.sydney.edu.au, dianqili@uw.edu Meng.Fang@liverpool.ac.uk, zhou@umiacs.umd.edu jgao95@stu.hit.edu.cn, zhanyibing@jd.com, dacheng.tao@gmail.com Abstract We present Twin Answer Sentences Attack (TASA), an adversarial attack method for ques- tion answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers. Despite phe- nomenal progress on general adversarial at- tacks, few works have investigated the vulner- ability and attack specifically for QA models. In this work, we first explore the biases in the existing models and discover that they mainly rely on keyword matching between the ques- tion and context, and ignore the relevant con- textual relations for answer prediction. Based on two biases above, TASA attacks the target model in two folds: (1) lowering the model’s confidence on the gold answer with a per- turbed answer sentence; (2) misguiding the model towards a wrong answer with a distract- ing answer sentence. Equipped with designed beam search and filtering methods, TASA can generate more effective attacks than existing textual attack methods while sustaining the quality of contexts, in extensive experiments on five QA datasets and human evaluations. 1 Introduction Question Answering (QA) is the cornerstone of various NLP tasks. In extractive QA (the most common setting), given a question and an associ- ated context, a QA model needs to comprehend on the context and predict the answer (Rajpurkar et al., 2016). While most works keep improving the answer correctness on benchmarks (Devlin et al., 2019; Yu et al., 2018), few studies investigate the robustness of QA models, e.g., is the performance achieved by sound contextual comprehension or via shortcuts like keyword matching? Although adversarial attacks attract growing interests in com- puter vision (Goodfellow et al., 2014; Zhao et al., 2018) and recently in NLP (Ren et al., 2019; Li ∗Work was done when Yu Cao was an intern at JD Ex- plore Academy. et al., 2021), most of them study general tasks with- out taking into account the properties of QA. The vulnerability and biases of models can lead to catas- trophic failures outside the benchmark datasets. An effective way to study them is through adversarial attacks specifically designed for QA tasks. Generating adversarial textual examples is chal- lenging due to the discrete syntactic restriction, es- pecially on QA, where the additional relationship between question and context should be further considered. Existing works such as AddSent and Human-in-the-loop (Jia and Liang, 2017; Wallace et al., 2019b) heavily rely on human annotators to create effective adversarial QA examples, which are costly and hard to scale. A few studies (Gan and Ng, 2019; Wang et al., 2020; Wallace et al., 2019a) can generate adversarial …

Content truncated to 500 words