Robust Question Answering Through Sub-part Alignment

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1251–1263 June 6–11, 2021. ©2021 Association for Computational Linguistics 1251 Robust Question Answering Through Sub-part Alignment Jifan Chen and Greg Durrett The University of Texas at Austin {jfchen, gdurrett}@cs.utexas.edu Abstract Current textual question answering (QA) mod- els achieve strong performance on in-domain test sets, but often do so by fitting surface- level patterns, so they fail to generalize to out- of-distribution settings. To make a more ro- bust and understandable QA system, we model question answering as an alignment problem. We decompose both the question and context into smaller units based on off-the-shelf se- mantic representations (here, semantic roles), and align the question to a subgraph of the context in order to find the answer. We for- mulate our model as a structured SVM, with alignment scores computed via BERT, and we can train end-to-end despite using beam search for approximate inference. Our use of ex- plicit alignments allows us to explore a set of constraints with which we can prohibit cer- tain types of bad model behavior arising in cross-domain settings. Furthermore, by inves- tigating differences in scores across different potential answers, we can seek to understand what particular aspects of the input lead the model to choose the answer without relying on post-hoc explanation techniques. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets. The results show that our model is more robust than the standard BERT QA model, and constraints derived from alignment scores allow us to ef- fectively trade off coverage and accuracy. 1 Introduction Current text-based question answering models learned end-to-end often rely on spurious patterns between the question and context rather than learn- ing the desired behavior. They may ignore the question entirely (Kaushik and Lipton, 2018), fo- cus primarily on the answer type (Mudrakarta et al., 2018), or otherwise bypass the “intended” mode of reasoning for the task (Chen and Durrett, 2019; Niven and Kao, 2019). Thus, these models are not robust to adversarial attacks (Jia and Liang, What day was Super Bowl 50 played on? The game was played on February 7, 2016 … Super Bowl 50 was an American football game to determine the champion … Adversarial sentence Sentence with correct answer Question The Champ Bowl was played on the day of August 18, 1991 Figure 1: A typical example on adversarial SQuAD. By breaking the question and context down into smaller units, we can expose the incorrect entity match and use explicit constraints to fix it. The solid lines denote edges from SRL and coreference, and the dotted lines denote the possible alignments between the arguments (desired in red, actual in black). 2017; Iyyer et al., 2018; Wallace et al., 2019): they can be fooled by surface-level distractor answers that follow the spurious patterns. Methods like ad- versarial training (Miyato et al., 2016; Wang and Bansal, 2018; Lee et al., 2019; Yang et …

Content truncated to 500 words