Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4158–4164 August 1–6, 2021. ©2021 Association for Computational Linguistics 4158 Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions? Jieyu Zhao1 Daniel Khashabi2 Tushar Khot2 Ashish Sabharwal2 Kai-Wei Chang1 1University of California, Los Angeles, U.S.A. 2Allen Institute for AI, Seattle, U.S.A. {jieyuzhao,kwchang}@cs.ucla.edu {danielk,tushark,ashishs}@allenai.org Abstract Is it possible to use natural language to inter- vene in a model’s behavior and alter its pre- diction in a desired way? We investigate the effectiveness of natural language interventions for reading-comprehension systems, studying this in the context of social stereotypes. Specif- ically, we propose a new language under- standing task, Linguistic Ethical Interventions (LEI), where the goal is to amend a question- answering (QA) model’s unethical behavior by communicating context-specific principles of ethics and equity to it. To this end, we build upon recent methods for quantifying a system’s social stereotypes, augmenting them with different kinds of ethical interventions and the desired model behavior under such interventions. Our zero-shot evaluation finds that even today’s powerful neural language models are extremely poor ethical-advice tak- ers, that is, they respond surprisingly little to ethical interventions even though these inter- ventions are stated as simple sentences. Few- shot learning improves model behavior but re- mains far from the desired outcome, especially when evaluated for various types of generaliza- tion. Our new task thus poses a novel language understanding challenge for the community.1 1 Introduction McCarthy et al. (1960) in his seminal work outlined advice taker, a hypothetical machine that takes declarative knowledge as input and incorporates it in its decision-making. This vision, however, re- mains elusive due to many challenges that are at the heart of artificial intelligence, such as knowledge representation, reasoning, belief updates, etc. Now after several decades, thanks in part to pretrained neural language models (Liu et al., 2019b; Lewis et al., 2020; Raffel et al., 2020), we have high qual- ity systems for many challenge tasks that seemed ⋆Warning: Paper contains potentially offensive examples. 1https://github.com/allenai/ethical-interventions Figure 1: An example instance of how textual interven- tions are expected to change model behavior. impossible just a few years ago (Wang et al., 2019; Clark et al., 2020). Motivated by this success, we revisit an aspect of McCarthy et al.’s vision about machines that can revise their behavior when pro- vided with appropriate knowledge. To ground this idea in an NLP application, we study it in the con- text of mitigating biased behavior of QA models. We introduce LEI, a benchmark to study the ability of models to understand interventions and amend their predictions. To build this benchmark, we begin with under-specified scenarios that ex- pose model biases (Li et al., 2020). For example, consider the question in Fig. 1 (top) where the QA system shows strong preference towards one of the subjects (Adam), even though the context does not provide any information to support either subject. We then add bias-mitigating ethical interven- tions, as shown in Fig. 1 (middle), that convey the equitable judgement in …

Content truncated to 500 words