Does it care what you asked? Understanding Importance of Verbs in Deep Learning QA System

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 322–324 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 322 Does it care what you asked? Understanding Importance of Verbs in Deep Learning QA System Barbara Rychalska∗1, Dominika Basaj∗2, Przemysław Biecek3, and Anna Wróblewska4 d.basaj, b.rychalska, p.biecek, a.wroblewska@mini.pw.edu.pl 1,2,3,4Warsaw University of Technology, Warsaw, Poland 2Tooploox, Wroclaw, Poland Abstract In this paper we present the results of an inve- stigation of the importance of verbs in a deep learning QA system trained on SQuAD data- set. We show that main verbs in questions carry little influence on the decisions made by the system - in over 90% of researched ca- ses swapping verbs for their antonyms did not change system decision. We track this pheno- menon down to the insides of the net, analy- zing the mechanism of self-attention and va- lues contained in hidden layers of RNN. Fi- nally, we recognize the characteristics of the SQuAD dataset as the source of the problem. Our work refers to the recently popular to- pic of adversarial examples in NLP, combined with investigating deep net structure. 1 Introduction Recent advances in interpretability for NLP focus on the problem of adversarial examples (Ribeiro et al., 2018) (Jia and Liang, 2017) which lead sys- tems to mistakenly change output. In case of QA systems, either questions or contexts are modified, and it is shown that seemingly small changes in semantics flip system decisions. In this paper we take a different approach: we create heavy differences in meaning by generating questions with their meaning negated, and obse- rve system outputs. Our initial hypothesis was that verbs together with nouns should be of paramount importance to the system, as they are the main cre- ators of meaning in language. We find that rever- sing verb meaning disturbs system output in 9.5% of cases, with little influence on decision certa- inty. We then proceed to explain this phenomenon by observing the behavior of deep net architecture and the characteristics of the SQuAD dataset (Raj- purkar et al., 2016) itself. As a basis of our research we use the QA system described in Chen et al. (2017). We pick this mo- *Both authors contributed equally. del for its good performance and state-of-the-art approach. 2 Negating Question Meaning The first step we take is to measure the impact of verb meaning in question on system output. First, we swap verbs in questions for their anto- nyms using WordNet (Miller, 1995). For auxiliary verbs, we insert their negations (e.g. is - isn’t). If an antonym is not found in WordNet, we substitute a random verb without assuring that its meaning matches the context. Examples of modified qu- estions are presented in Table 1. Next, we test how many system outputs for original questions match the outputs for questions with reversed meaning of verbs. As matching we classify identical answers and also some cases with minimal differences in meaning (where we are sure that the …

Content truncated to 500 words