Adversarial Examples for Evaluating Reading Comprehension Systems

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021–2031 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Adversarial Examples for Evaluating Reading Comprehension Systems Robin Jia Computer Science Department Stanford University robinjia@cs.stanford.edu Percy Liang Computer Science Department Stanford University pliang@cs.stanford.edu Abstract Standard accuracy metrics indicate that reading comprehension systems are mak- ing rapid progress, but the extent to which these systems truly understand language remains unclear. To reward systems with real language understanding abili- ties, we propose an adversarial evalua- tion scheme for the Stanford Question An- swering Dataset (SQuAD). Our method tests whether systems can answer ques- tions about paragraphs that contain adver- sarially inserted sentences, which are au- tomatically generated to distract computer systems without changing the correct an- swer or misleading humans. In this ad- versarial setting, the accuracy of sixteen published models drops from an average of 75% F1 score to 36%; when the ad- versary is allowed to add ungrammatical sequences of words, average accuracy on four models decreases further to 7%. We hope our insights will motivate the de- velopment of new models that understand language more precisely. 1 Introduction Quantifying the extent to which a computer sys- tem exhibits intelligent behavior is a longstanding problem in AI (Levesque, 2013). Today, the stan- dard paradigm is to measure average error across a held-out test set. However, models can succeed in this paradigm by recognizing patterns that hap- pen to be predictive on most of the test examples, while ignoring deeper, more difficult phenomena (Rimell et al., 2009; Paperno et al., 2016). In this work, we propose adversarial evaluation for NLP, in which systems are instead evaluated on adversarially-chosen inputs. We focus on the Article: Super Bowl 50 Paragraph: “Peyton Manning became the first quarter- back ever to lead two different teams to multiple Super Bowls. He is also the oldest quarterback ever to play in a Super Bowl at age 39. The past record was held by John Elway, who led the Broncos to victory in Super Bowl XXXIII at age 38 and is currently Denver’s Execu- tive Vice President of Football Operations and General Manager. Quarterback Jeff Dean had jersey number 37 in Champ Bowl XXXIV.” Question: “What is the name of the quarterback who was 38 in Super Bowl XXXIII?” Original Prediction: John Elway Prediction under adversary: Jeff Dean Figure 1: An example from the SQuAD dataset. The BiDAF Ensemble model originally gets the answer correct, but is fooled by the addition of an adversarial distracting sentence (in blue). SQuAD reading comprehension task (Rajpurkar et al., 2016), in which systems answer questions about paragraphs from Wikipedia. Reading com- prehension is an appealing testbed for adversarial evaluation, as existing models appear successful by standard average-case evaluation metrics: the current state-of-the-art system achieves 84.7% F1 score, while human performance is just 91.2%.1 Nonetheless, it seems unlikely that existing sys- tems possess true language understanding and rea- soning capabilities. Carrying out adversarial evaluation on SQuAD requires new methods …

Content truncated to 500 words