The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6999–7013 August 1–6, 2021. ©2021 Association for Computational Linguistics 6999 The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity David Gros Computer Science Dept. University of California, Davis dgros@ucdavis.edu Yu Li Computer Science Dept. University of California, Davis yooli@ucdavis.edu Zhou Yu Computer Science Dept. Columbia University zy2461@columbia.edu Abstract Humans are increasingly interacting with ma- chines through language, sometimes in con- texts where the user may not know they are talking to a machine (like over the phone or a text chatbot). We aim to understand how system designers and researchers might allow their systems to confirm its non-human iden- tity. We collect over 2,500 phrasings related to the intent of “Are you a robot?". This is paired with over 2,500 adversarially selected utterances where only confirming the system is non-human would be insufficient or disflu- ent. We compare classifiers to recognize the in- tent and discuss the precision/recall and model complexity tradeoffs. Such classifiers could be integrated into dialog systems to avoid undesired deception. We then explore how both a generative research model (Blender) as well as two deployed systems (Amazon Alexa, Google Assistant) handle this intent, finding that systems often fail to confirm their non- human identity. Finally, we try to understand what a good response to the intent would be, and conduct a user study to compare the im- portant aspects when responding to this intent. 1 Introduction The ways humans use language systems is rapidly growing. There are tens of thousands of chatbots on platforms like Facebook Messenger and Mi- crosoft’s Skype (Brandtzaeg and Følstad, 2017), and millions of smart speakers in homes (Olson and Kemery, 2019). Additionally, systems such as Google’s Duplex (Leviathan and Matias, 2018), which phone calls businesses to make reservations, foreshadows a future where users might have un- solicited conversations with human sounding ma- chines over the phone. This future creates many challenges (Følstad and Brandtzæg, 2017; Henderson et al., 2018). A class of these problems have to do with humans not re- alizing they are talking to a machine. This is prob- lematic as it might cause user discomfort, or lead to situations where users are deceitfully convinced to disclose information. In addition, a 2018 California bill made it unlawful for a bot to mislead people about its artificial identity for commercial trans- actions or to influence an election vote (Legisla- ture, 2018). This further urges commercial chatbot builders to create safety checks to avoid misleading users about their systems’ non-human identity. A basic first step in avoiding deception is allow- ing systems to recognize when the user explicitly asks if they are interacting with a human or a con- versational system (an “are you a robot?" intent). There are reasons to think this might be difficult. For one, there are varied number of ways to convey this intent: When recognizing this …

Content truncated to 500 words