Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3266 - 3284 December 7-11, 2022 ©2022 Association for Computational Linguistics Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems David Gros∗ University of California, Davis dgros@ucdavis.edu Yu Li∗ Columbia University yl5016@columbia.edu Zhou Yu Columbia University zy2461@columbia.edu Abstract Dialog systems are often designed or trained to output human-like responses. However, some responses may be impossible for a machine to truthfully say (e.g. “that movie made me cry”). Highly anthropomorphic responses might make users uncomfortable or implicitly deceive them into thinking they are interacting with a human. We collect human ratings on the feasibility of approximately 900 two-turn dialogs sampled from 9 diverse data sources. Ratings are for two hypothetical machine embodiments: a fu- turistic humanoid robot and a digital assistant. We find that for some data-sources commonly used to train dialog systems, 20-30% of utter- ances are not viewed as possible for a machine. Rating is marginally affected by machine em- bodiment. We explore qualitative and quan- titative reasons for these ratings. Finally, we build classifiers and explore how modeling con- figuration might affect output permissibly, and discuss implications for building less falsely anthropomorphic dialog systems. 1 Introduction At the 1939 Worlds Fair, the Westinghouse Electric Corporation debuted their latest humanoid robot, Elektro, with many tricks that wowed spectators. On stage, it could respond to voice commands spo- ken into a telephone, talk through a record player, and walk a short distance. In one exciting part of the show, Elektro would say that he would like a cigarette. An attendant would place a cigarette in his mechanical jaws, give it a light, and when in- structed, Elektro would pump air to give it a few puffs (Williams, 2019). While the thought of a walking, talking robot might seem normal, this last trick of a robot wanting to smoke would likely sur- prise even modern audiences. Over 80 years later, we live in a world with mil- lions of talking machines, creating perplexing ques- tions on what these machines can and should do. With recent advances in data-driven NLP, human- level dialog agents seem potentially within reach (Adiwardana et al., 2020; Chowdhery et al., 2022). Yet, human conversation and machine conver- sation are not the same. Whether it is enjoying cigarettes, talking about their daughter’s favorite pizza, or sharing a touching story that made them cry — some utterances are possible for a human to say, but not a machine. *Core contributors. Description of contributions appx A Figure 1: The first page for survey data collection in a humanoid embodiment 3266 But why is this, and what should the future of dialog systems be? We seek to add to progress on this by exploring the following research questions: RQ1. What is the distribution of impermissibly anthropomorphic utterances in common data sources? Current techniques are data-driven, so understanding anthropomorphism in the input data might inform system results §5 RQ2. What factors determine how a user might rate an utterance? Understanding …

Content truncated to 500 words