Evaluating Neural Model Robustness for Machine Comprehension

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2470–2481 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2470 Evaluating Neural Model Robustness for Machine Comprehension Winston Wu Center for Language and Speech Processing Department of Computer Science Johns Hopkins University wswu@jhu.edu Dustin Arendt1 Svitlana Volkova2 1Visual Analytics Group 2Data Sciences and Analytics Group Pacific Northwest National Laboratory first.last@pnnl.gov Abstract We evaluate neural model robustness to adver- sarial attacks using different types of linguistic unit perturbations – character and word, and propose a new method for strategic sentence- level perturbations. We experiment with dif- ferent amounts of perturbations to examine model confidence and misclassification rate, and contrast model performance with differ- ent embeddings BERT and ELMo on two benchmark datasets SQuAD and TriviaQA. We demonstrate how to improve model perfor- mance during an adversarial attack by using ensembles. Finally, we analyze factors that af- fect model behavior under adversarial attack, and develop a new model to predict errors dur- ing attacks. Our novel findings reveal that (a) unlike BERT, models that use ELMo embed- dings are more susceptible to adversarial at- tacks, (b) unlike word and paraphrase, char- acter perturbations affect the model the most but are most easily compensated for by adver- sarial training, (c) word perturbations lead to more high-confidence misclassifications com- pared to sentence- and character-level pertur- bations, (d) the type of question and model an- swer length (the longer the answer the more likely it is to be incorrect) is the most pre- dictive of model errors in adversarial setting, and (e) conclusions about model behavior are dataset-specific. 1 Introduction Deep neural models have recently gained popular- ity, leading to significant improvements in many Natural Language Understanding (NLU) tasks (Goldberg, 2017). However, the research com- munity still lacks in-depth understanding of how these models work and what kind of linguistic in- formation is actually captured by neural networks (Feng et al., 2018). Evaluating model robustness to manipulated inputs and analyzing model behav- ior during adversarial attacks can provide deeper Context: One of the most famous people born in War- saw was Maria Skłodowska-Curie, who achieved inter- national recognition for her research on radioactivity and was the first female recipient of the Nobel Prize. Question: What was Maria Curie the first female recipi- ent of? Answer: Nobel Prize Table 1: Example MC question from SQuAD. insights into how much language understanding models actually have (Hsieh et al., 2019; Si et al., 2020). Moreover, as has been widely discussed, models should be optimized not only for accuracy but also for other important criteria such as relia- bility, accountability and interpretability (Lipton, 2018; Doshi-Velez and Kim, 2017; Ribeiro et al., 2016; Goodman and Flaxman, 2017). In this work, we evaluate neural model robust- ness on machine comprehension (MC), a task de- signed to measure a system’s understanding of text. In this task, given a context paragraph and a ques- tion, the machine is tasked to provide an answer. We focus …

Content truncated to 500 words