On the Transferability of Minimal Prediction Preserving Inputs in Question Answering

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1288–1300 June 6–11, 2021. ©2021 Association for Computational Linguistics 1288 On the Transferability of Minimal Prediction Preserving Inputs in Question Answering Shayne Longpre∗ Apple Inc. slongpre@apple.com Yi Lu∗ Apple Inc. ylu7@apple.com Christopher DuBois Apple Inc. cdubois@apple.com Abstract Recent work (Feng et al., 2018) establishes the presence of short, uninterpretable input frag- ments that yield high confidence and accuracy in neural models. We refer to these as Min- imal Prediction Preserving Inputs (MPPIs). In the context of question answering, we in- vestigate competing hypotheses for the exis- tence of MPPIs, including poor posterior cal- ibration of neural models, lack of pretrain- ing, and “dataset bias" (where a model learns to attend to spurious, non-generalizable cues in the training data). We discover a per- plexing invariance of MPPIs to random train- ing seed, model architecture, pretraining, and training domain. MPPIs demonstrate remark- able transferability across domains — achiev- ing significantly higher performance than com- parably short queries. Additionally, penalizing over-confidence on MPPIs fails to improve ei- ther generalization or adversarial robustness. These results suggest the interpretability of MPPIs is insufficient to characterize general- ization capacity of these models. We hope this focused investigation encourages more system- atic analysis of model behavior outside of the human interpretable distribution of examples. 1 Introduction Feng et al. (2018) establish the presence of short- ened input sequences that yield high confidence and accuracy for non-pretrained neural models. These Minimal Prediction Preserving Inputs (MPPIs) are constructed by iteratively removing the least im- portant word from the query to obtain the shortest sequence for which the model’s prediction remains unchanged (example shown in Figure 1).1 Humans are unable to make either confident or accurate pre- dictions on these inputs. Follow up work treats ∗ equal contribution 1For question answering we construct MPPIs by only removing words from the query. Modifying the context para- graph is poorly defined in MPPI generation as it perturbs the output space, rendering an answer impossible or trivial. SQUAD Context ... The site currently houses three cinemas, including the restored Classic the United Kingdom’s last surviving news cinema still in full-time operation—alongside two new screens ... Original What’s the name of the United Kingdom ’s sole remaining news cinema ? Reduced news Confidence 0.57 → 0.51 Figure 1: A SQUAD dev set example. Given the orig- inal Context, the model makes the same correct predic- tion (“Classic”) on the Reduced question (MPPI) as the Original, with almost the same score. For humans, the reduced question, “news”, is nonsensical. strong model performance on such partial-inputs as equivalent with models improperly learning the task (Feng et al., 2019; Kaushik and Lipton, 2018; He et al., 2019). Accordingly, we evaluate this proposition in question answering (QA), investigat- ing the properties of MPPIs and how their exis- tence relates to “dataset bias", out-of-domain gen- eralization, and adversarial robustness. First we examine the hypothesis that MPPIs are a symptom …

Content truncated to 500 words