Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6707–6723 August 1–6, 2021. ©2021 Association for Computational Linguistics 6707 Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models Tongshuang Wu1 Marco Tulio Ribeiro2 Jeffrey Heer1 Daniel S. Weld1,3 1University of Washington 2Microsoft Research 3Allen Institute for Artificial Intelligence wtshuang@cs.uw.edu marcotcr@microsoft.com {jheer,weld}@cs.uw.edu Abstract While counterfactual examples are useful for analysis and training of NLP models, cur- rent generation methods either rely on man- ual labor to create very few counterfactuals, or only instantiate limited types of perturba- tions such as paraphrases or word substitutions. We present Polyjuice, a general-purpose coun- terfactual generator that allows for control over perturbation types and locations, trained by finetuning GPT-2 on multiple datasets of paired sentences. We show that Polyjuice pro- duces diverse sets of realistic counterfactuals, which in turn are useful in various distinct applications: improving training and evalua- tion on three different tasks (with around 70% less annotation effort than manual generation), augmenting state-of-the-art explanation tech- niques, and supporting systematic counterfac- tual error analysis by revealing behaviors eas- ily missed by human experts. 1 Introduction Counterfactual reasoning — mentally simulating what would have happened if conditions were dif- ferent — is a common tool for making causality as- sessments (Kahneman and Tversky, 1981), which in turn are crucial for model evaluation, error anal- ysis, and explanation (Miller, 2019). For example, in Figure 1, “It is great for kids” is perturbed into multiple variations, each providing unique insights by simulating what would have happened if the sentence was different. Applications of counterfactual reasoning to NLP generally specify the relationship x ) ˆx, and then create ˆx according to the relationship. As a re- sult, prior work has tailored counterfactual gen- erators for different applications, only collecting subsets of ˆx that are useful for the specific task. For example, to support model training and eval- uation, human annotators create counterfactuals It is great for kids. It is great for kids→adults. It is great→scary for kids. delete lexical negation It is great for kids. Polyjuice generates ̂xi Original x Select for use cases B A C f( ̂xi) — + + + + f(x) + It is great for kids Training Evaluation Explanation D Error Analysis It is not great for kids. It is great for kids→no one. Figure 1: Overview: (A) given a sentiment analysis in- stance x, Polyjuice1generates (B) various counterfactu- als ˆx, which are then (C) selected for downstream use. e.g., in (D) we select counterfactual explanations that complement a black box explanation: though “great” and “kids” are deemed important, perturbing them may not affect the prediction f(x) = f(ˆx) = positive, reveal- ing model failures not covered by feature attributions. that change the groundtruth labels by manually rewriting instances (Gardner et al., 2020; Qin et al., 2019) or defining perturbation functions (Ribeiro et al., 2020). Manual rewrites are costly (e.g., 4–5 minutes per counterfactual (Kaushik et al., 2020)) and …

Content truncated to 500 words