Towards Interpreting BERT for Reading Comprehension Based QA

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 3236–3242, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 3236 Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath∗, Preksha Nema, Deep Sahni, Mitesh M. Khapra Robert Bosch Centre for Data Science and AI (RBC-DSAI) Indian Institute of Technology Madras, Chennai, India {sahanjich,deep.sahani11}@gmail.com, {preksha,miteshk}@cse.iitm.ac.in Abstract BERT and its variants have achieved state- of-the-art performance in various NLP tasks. Since then, various works have been proposed to analyze the linguistic information being cap- tured in BERT. However, the current works do not provide an insight into how BERT is able to achieve near human-level performance on the task of Reading Comprehension based Question Answering. In this work, we attempt to interpret BERT for RCQA. Since BERT lay- ers do not have predefined roles, we define a layer’s role or functionality using Integrated Gradients. Based on the defined roles, we per- form a preliminary analysis across all layers. We observed that the initial layers focus on query-passage interaction, whereas later lay- ers focus more on contextual understanding and enhancing the answer prediction. Specif- ically for quantifier questions (how much/how many), we notice that BERT focuses on con- fusing words (i.e., on other numerical quan- tities in the passage) in the later layers, but still manages to predict the answer correctly. The fine-tuning and analysis scripts will be publicly available at https://github.com/ iitmnlp/BERT-Analysis-RCQA. 1 Introduction The past decade has witnessed a surge in the de- velopment of deep neural network models to solve NLP tasks. Pretrained language models such as ELMO (Peters et al., 2018a), BERT (Devlin et al., 2018) , XLNet (Yang et al., 2019) etc. have achieved state-of-the-art results on various NLP tasks. This success motivated various studies to un- derstand how BERT achieves human-level perfor- mance on these tasks. Tenney et al. (2019); Peters et al. (2018b) analyze syntactic and semantic roles played by different layers in such models. Clark et al. (2019) specifically analyze BERT’s attention ∗* now at Google Research, Bangalore heads for syntactic and linguistic phenomena. Most of these works focus on tasks such as sentiment classification, syntactic/semantic tags prediction, natural language inference, and so on. However, to the best of our knowledge, BERT has not been thoroughly analyzed for complex tasks like RCQA. It is a challenging task because of 1) the large num- ber of parameters and non-linearities in BERT, and 2) the absence of pre-defined roles across layers in BERT as compared to pre-BERT models like BiDAF (Seo et al., 2016) or DCN (Xiong et al., 2016). In this work, we take the first step to iden- tify each layer’s role using the attribution method of Integrated Gradients (Sundararajan et al., 2017). We then try to map these roles to the following functions, deemed necessary in pre-BERT models to reach the answer: (i) learn contextual representa- tions for the passage and the question, individually, (ii) attend to information in the passage specific to the question and, (iii) predict the answer. …

Content truncated to 500 words