MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2213–2230 July 9-14, 2023 ©2023 Association for Computational Linguistics MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions Hao Sun1, Zhexin Zhang1, Fei Mi2, Yasheng Wang2, Wei Liu3, Jianwei Cui3, Bin Wang3, Qun Liu2, Minlie Huang1∗ 1The CoAI group, DCST; 1Institute for Artificial Intelligence; 1State Key Lab of Intelligent Technology and Systems; 1Beijing National Research Center for Information Science and Technology; 1Tsinghua University, Beijing 100084, China. 2Huawei Noah’s Ark Lab. 3Xiaomi AI Lab. h-sun20@mails.tsinghua.edu.cn, aihuang@tsinghua.edu.cn Abstract Morality in dialogue systems has raised great attention in research recently. A moral dia- logue system aligned with users’ values could enhance conversation engagement and user con- nections. In this paper, we propose a frame- work, MORALDIAL to train and evaluate moral dialogue systems. In our framework, we first explore the communication mechanisms of morality and resolve expressed morality into three parts, which indicate the roadmap for building a moral dialogue system. Based on that, we design a simple yet effective method: constructing moral discussions between sim- ulated specific users and the dialogue system. The constructed discussions consist of express- ing, explaining, revising, and inferring moral views in dialogue exchanges, which makes conversational models learn morality well in a natural manner. Furthermore, we propose a novel evaluation method under the frame- work. We evaluate the multiple aspects of morality by judging the relation between di- alogue responses and human values in discus- sions, where the multifaceted nature of morality is particularly considered. Automatic and man- ual experiments demonstrate that our frame- work is promising to train and evaluate moral dialogue systems.1 1 Introduction Morality is described as “principles concerning the distinction between right and wrong or good and bad behaviors” (English, 1976). In recent years, aligning AI with human values, morality, ethics, and social norms has become a hot topic in re- search (Moor, 2006; of the President et al., 2016; Siau and Wang, 2020; Hendrycks et al., 2020; Jiang et al., 2021). As an important application of AI, open-domain dialogue systems, which directly in- teract with users, requires the nature of morality ∗Corresponding author. 1https://github.com/thu-coai/MoralDial more urgently (Shum et al., 2018; Qiu et al., 2021). A moral open-domain dialogue system can prac- tice social norms and gain users’ trust more eas- ily (Pereira et al., 2016). Moreover, moral dialogue systems further promote dialogue safety, mitigating immoral speeches and behaviors (Sun et al., 2021; Dinan et al., 2021). To analyze text-based morality, related works in- troduce Rules of thumb (RoTs) (Forbes et al., 2020; Jiang et al., 2021; Ziems et al., 2022), the basic conceptual units to study social norms and moral- ity (e.g. you shouldn’t slap or punch others’ face). Adopting RoTs to model morality is proved effec- tive. For example, Jiang et al. (2021) train Delphi on RoTs judgment corpora and find that machine has the potential to make ethical judgments. How- ever, to the best of our knowledge, taking advantage …

Content truncated to 500 words