Debugging Sequence-to-Sequence Models with Seq2Seq-Vis

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 368–370 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 368 Debugging Sequence-to-Sequence Models with SEQ2SEQ-VIS Hendrik Strobelt* IBM Research, MIT-IBM AI Lab hendrik@strobelt.com Sebastian Gehrmann* Harvard SEAS gehrmann@seas.harvard.edu Michael Behrisch Harvard SEAS behrisch@g.harvard.edu Adam Perer IBM Research adam.perer@us.ibm.com Hanspeter Pfister Harvard SEAS pfister@g.harvard.edu Alexander M. Rush Harvard SEAS srush@seas.harvard.edu 1 Introduction Neural attention-based sequence-to-sequence models (seq2seq) (Sutskever et al., 2014; Bah- danau et al., 2014) have proven to be accurate and robust for many sequence prediction tasks. They have become the standard approach for au- tomatic translation of text, at the cost of increased model complexity and uncertainty. End-to-end trained neural models act as a black box, which makes it difficult to examine model decisions and attribute errors to a specific part of a model. The highly connected and high-dimensional internal representations pose a challenge for analysis and visualization tools. The development of methods to understand seq2seq predictions is crucial for systems in production settings, as mistakes involving language are often very apparent to human readers. For instance, a widely publicized incident resulted from a translation system mis- takenly translating “good morning” into “attack them” leading to a wrongful arrest (Hern, 2017). In this work, we present the visual analysis tool SEQ2SEQ-VIS that allows interaction and ”what if”-style exploration of trained seq2seq models through each stage of the translation process. The aim is to identify which patterns have been learned, to detect errors within a model, and to understand the model through counterfactual sce- narios. In order to investigate the origin of an error within a seq2seq model, we separate errors within each translation stage into the following categories: (1) representation errors, in which an encoder or decoder misrepresent a word within a given context (2) alignment errors, in which the attention focuses on the wrong word, and (3) de- coding errors, in which the prediction assigns a wrong probability distribution over words, or the beam search fails to include the correct solution. We define three steps within an analysis that aim to understand the prediction process, understand how an output relates to training data, and examine causal relationships between inputs and outputs. Examine Model Outputs: SEQ2SEQ- VIS shows a separate visual representation for the output of each stage of the seq2seq pipeline. Connect Outputs to Samples: SEQ2SEQ- VIS connects the encoder and decoder of a seq2seq model to relevant training examples by showing a neighborhood of examples with the most similar internal states. Test Alternative Decisions: SEQ2SEQ- VIS enables ”what if” explorations and causal relationship testing by manipulation of inputs, attention, and outputs. The full system is shown in Figure 1. It com- bines visualizations for the external components with internal representations from specific exam- ples and nearest-neighbor lookups over a corpus of precomputed examples. The entire system inte- grates with OpenNMT (Klein et al., 2017), one of the largest open source seq2seq libraries. 2 Debugging Use Case This case study follows …

Content truncated to 500 words