Undersensitivity in Neural Reading Comprehension

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1152–1165 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 1152 Undersensitivity in Neural Reading Comprehension Johannes Welbl†∗ Pasquale Minervini† Max Bartolo† Pontus Stenetorp† Sebastian Riedel† †University College London {j.welbl,p.minervini,m.bartolo,p.stenetorp,s.riedel}@cs.ucl.ac.uk Abstract Current reading comprehension methods gen- eralise well to in-distribution test sets, yet perform poorly on adversarially selected data. Prior work on adversarial inputs typically stud- ies model oversensitivity: semantically invari- ant text perturbations that cause a model’s pre- diction to change. Here we focus on the com- plementary problem: excessive prediction un- dersensitivity, where input text is meaning- fully changed but the model’s prediction does not, even though it should. We formulate an adversarial attack which searches among se- mantic variations of the question for which a model erroneously predicts the same answer, and with even higher probability. We demon- strate that models trained on both SQuAD2.0 and NewsQA are vulnerable to this attack, and then investigate data augmentation and adver- sarial training as defences. Both substantially decrease adversarial vulnerability, which gen- eralises to held-out data and held-out attack spaces. Addressing undersensitivity further- more improves model robustness on the pre- viously introduced ADDSENT and ADDONE- SENT datasets, and models generalise better when facing train/evaluation distribution mis- match: they are less prone to overly rely on shallow predictive cues present only in the training set, and outperform a conventional model by as much as 10.9% F1. 1 Introduction Neural networks can be vulnerable to adversar- ial input perturbations (Szegedy et al., 2013; Ku- rakin et al., 2016). In Natural Language Pro- cessing (NLP), which operates on discrete sym- bol sequences, adversarial attacks can take a vari- ety of forms (Ettinger et al., 2017; Alzantot et al., 2018) including character perturbations (Ebrahimi ∗Now at DeepMind. et al., 2018), semantically invariant reformula- tions (Ribeiro et al., 2018b; Iyyer et al., 2018a) or – specifically in Reading Comprehension (RC) – ad- versarial text insertions (Jia and Liang, 2017; Wang and Bansal, 2018). A model’s inability to handle adversarially chosen input text puts into perspec- tive otherwise impressive generalisation results for in-distribution test sets (Seo et al. (2017); Yu et al. (2018); Devlin et al. (2019); inter alia) and con- stitutes an important caveat to conclusions drawn regarding a model’s comprehension abilities. While semantically invariant text transforma- tions can remarkably alter a model’s predictions, the complementary problem of model undersen- sitivity is equally troublesome: a model’s text in- put can often be drastically changed in meaning while retaining the original prediction. In particular, previous works (Feng et al., 2018; Ribeiro et al., 2018a; Welbl et al., 2020) show that even after deletion of all but a small fraction of input words, models often produce the same output. However, such reduced inputs are usually unnatural to a hu- man reader, and it is both unclear what behaviour we should expect from natural language models evaluated on unnatural text, and how to use such unnatural inputs to improve models. In this work we explore RC …

Content truncated to 500 words