Learning to Explain Selectively: A Case Study on Question Answering

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8372 - 8382 December 7-11, 2022 ©2022 Association for Computational Linguistics Learning to Explain Selectively: A Case Study on Question Answering Shi Feng University of Chicago shif@uchicago.edu Jordan Boyd-Graber University of Maryland jbg@umiacs.umd.edu Abstract Explanations promise to bridge the gap be- tween humans and AI, yet it remains diffi- cult to achieve consistent improvement in AI- augmented human decision making. The use- fulness of AI explanations depends on many factors, and always showing the same type of explanation in all cases is suboptimal—so is relying on heuristics to adapt explanations for each scenario. We propose learning to explain “selectively”: for each decision that the user makes, we use a model to choose the best ex- planation from a set of candidates and update this model with feedback to optimize human performance. We experiment on a question answering task, Quizbowl, and show that selec- tive explanations improve human performance for both experts and crowdworkers. 1 Introduction Recent advances in machine learning (ML) (Sil- ver et al., 2017; Brown et al.; Jumper et al., 2021) sparked new interest in intelligence augmenta- tion—the vision that computers are not mere number-crunching tools but also interactive sys- tems that can augment humans at problem solv- ing and decision making (Engelbart, 1962). The hope is to combine the complementary strengths of human and machine, and to fully harness these models with human intuitions and oversight (Dafoe et al., 2020; Amodei et al., 2016). But this agenda is obstructed by the many counterintuitive traits of neural networks (NNs) (Szegedy et al., 2014; Good- fellow et al., 2015; Zhang et al., 2017) and our lack of theoretical understanding (Belkin et al., 2019): these models are not interpretable to humans by default, and it is difficult to foresee when they will fail. This lack of interpretability also amplifies the risk of model bias (Angwin et al., 2016; Bolukbasi et al., 2016; Caliskan et al., 2017), making it diffi- cult to use NN-powered AIs in real-world decision making (Poursabzi-Sangdeh et al., 2021). To bridge the gap between human and machine, various methods attempt to explain model predic- tions in human-interpretable terms, e.g., by pro- viding more context to the model’s uncertainty estimation (Gal et al., 2016; Bhatt et al., 2021), by highlighting the most important part of the in- put (Ribeiro et al., 2016; Lundberg and Lee, 2017; Ebrahimi et al., 2017), and by retrieving relevant training examples (Renkl, 2014; Koh and Liang, 2017). Grounded in psychology (Lombrozo, 2006; Kulesza et al., 2012), these explanations promise to augment human decision making (Pu and Chen, 2006; Rader et al., 2018). But in application- grounded evaluations—with real problems and real humans (Doshi-Velez and Kim, 2018), it proves difficult for any single explanation method to achieve consistent improvement in disparate con- texts (Bansal et al., 2021; Buçinca et al., 2020). A major contributor to this problem is the breadth of context that the explanation methods are applied in. Internally, the …

Content truncated to 500 words