Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6925–6940 August 1–6, 2021. ©2021 Association for Computational Linguistics 6925 Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution Jiacheng Xu and Greg Durrett Department of Computer Science The University of Texas at Austin {jcxu,gdurrett}@cs.utexas.edu Abstract Despite the prominence of neural abstractive summarization models, we know little about how they actually form summaries and how to understand where their decisions come from. We propose a two-step method to interpret summarization model decisions. We first an- alyze the model’s behavior by ablating the full model to categorize each decoder decision into one of several generation modes: roughly, is the model behaving like a language model, is it relying heavily on the input, or is it some- where in between? After isolating decisions that do depend on the input, we explore inter- preting these decisions using several different attribution methods. We compare these tech- niques based on their ability to select content and reconstruct the model’s predicted token from perturbations of the input, thus revealing whether highlighted attributions are truly im- portant for the generation of the next token. While this machinery can be broadly useful even beyond summarization, we specifically demonstrate its capability to identify phrases the summarization model has memorized and determine where in the training pipeline this memorization happened, as well as study com- plex generation phenomena like sentence fu- sion on a per-instance basis. 1 Introduction Transformer-based neural summarization models (Liu and Lapata, 2019; Stiennon et al., 2020; Xu et al., 2020b; Desai et al., 2020), especially pre- trained abstractive models like BART (Lewis et al., 2020) and PEGASUS (Zhang et al., 2020), have made great strides in recent years. These models demonstrate exciting new capabilities in terms of abstraction, but little is known about how these models work. In particular, do token generation decisions leverage the source text, and if so, which parts? Or do these decisions arise based primar- ily on knowledge from the language model (Jiang et al., 2020; Carlini et al., 2020), learned during pre-training or fine-tuning? Having tools to ana- lyze these models is crucial to identifying and fore- stalling problems in generation, such as toxicity (Gehman et al., 2020) or factual errors (Kryscinski et al., 2020; Goyal and Durrett, 2020, 2021). Although interpreting classification models for NLP has been widely studied from perspectives like feature attribution (Ribeiro et al., 2016; Sun- dararajan et al., 2017) and influence functions (Koh and Liang, 2017; Han et al., 2020), summarization specifically introduces some additional elements that make these techniques hard to apply directly. First, summarization models make sequential de- cisions from a very large state space. Second, encoder-decoder models have a special structure, featuring a complex interaction of decoder-side and encoder-side computation to select the next word. Third, pre-trained LMs blur the distinction between relying on implicit prior knowledge or explicit instance-dependent input. This paper aims to more fully interpret …

Content truncated to 500 words