BabyBERTa: Learning More Grammar With Small-Scale Child-Directed Language

Proceedings of the 25th Conference on Computational Natural Language Learning (CoNLL), pages 44–57 November 10–11, 2021. ©2021 Association for Computational Linguistics 44 “It seemed like an annoying woman”: On the Perception and Ethical Considerations of Affective Language in Text-Based Conversational Agents Lindsey Vanderlyn, Gianna Weber, Michael Neumann, Dirk V¨ath, Sarina Meyer, Ngoc Thang Vu Institute for Natural Language Processing University of Stuttgart, Germany lindsey.vanderlyn@ims.uni-stuttgart.de Abstract Previous research has found that task-oriented conversational agents are perceived more posi- tively by users when they provide information in an empathetic manner compared to a plain, emotionless information exchange. However, users’ perception and ethical considerations re- lated to a dialog systems’ response language style have received comparatively little atten- tion in the field of human-computer interac- tion. To bridge this gap, we explored these eth- ical implications through a scenario-based user study. 127 participants interacted with one of three variants of an affective, task-oriented conversational agent, each variant providing responses in a different language style. Af- ter the interaction, participants filled out a sur- vey about their feelings during the experiment and their perception of various aspects of the chatbot. Based on statistical and qualitative analysis of the responses, we found language style played an important role in how human- like participants perceived a dialog agent as well as how likable. Language style also had a direct effect on how users perceived the use of personal pronouns ‘I’ and ‘You’ and how they projected gender onto the chatbot. Finally, we identify and discuss ethical im- plications. In particular we focus on what factors/stereotypes influenced participants’ im- pressions of gender, and what trade-offs a more human-like chatbot brings. 1 Introduction Conversational Agents (CAs) have been an area of interest for many years. As technology has ad- vanced, these devices have become more capable and further embedded in users’ daily lives. With the advent of personal assistants, like Apple’s Siri and Amazon’s Alexa, focus has begun shifting away from simply conveying correct information to ad- ditionally considering the user experience (Yang et al., 2019) and how that information is conveyed. Rather than seeing a CA as an inanimate tool, users tend to view them as social actors (Nass et al., 1994) and infer or assign them personality traits. These perceived personas, whether intentionally de- signed into the CA or not, can influence how users perceive and interact with these systems. There- fore much research has gone into how to design appropriate personas (Kim et al., 2019; Nass et al., 1995) and investigate how users interpret them (Lee et al., 2019). However, while there is a study ex- ploring ethical issues in text-based affective dialog systems (Kretzschmar et al., 2019), comparatively little research has been done in this area. Open ethical questions thus remain unanswered around how users perceive and react to different dialog system designs. In this paper, we are particu- larly interested in how users perceive and emotion- ally react to a system mimicking human thoughts or emotional awareness, as well as what biases users …

Content truncated to 500 words