Improving Multi-label Malevolence Detection in Dialogues through Multi-faceted Label Correlation Enhancement

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 3543 - 3555 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Improving Multi-label Malevolence Detection in Dialogues through Multi-faceted Label Correlation Enhancement Yangjun Zhang1, Pengjie Ren2∗, Wentao Deng2, Zhumin Chen2, Maarten de Rijke1 1University of Amsterdam, 2Shandong University 1{y.zhang6, m.derijke}@uva.nl, 2{renpengjie, wentao.deng, chenzhumin}@sdu.edu.cn Abstract A dialogue response is malevolent if it is grounded in negative emotions, inappropriate behavior, or an unethical value basis in terms of content and dialogue acts. The detection of malevolent dialogue responses is attracting growing interest. Current research on detect- ing dialogue malevolence has limitations in terms of datasets and methods. First, available dialogue datasets related to malevolence are la- beled with a single category, but in practice assigning a single category to each utterance may not be appropriate as some malevolent utterances belong to multiple labels. Second, current methods for detecting dialogue malev- olence neglect label correlation. Therefore, we propose the task of multi-label dialogue malev- olence detection and crowdsource a multi- label dataset, multi-label dialogue malevo- lence detection (MDMD) for evaluation. We also propose a multi-label malevolence detec- tion model, multi-faceted label correlation en- hanced CRF (MCRF), with two label correla- tion mechanisms, label correlation in taxon- omy (LCT) and label correlation in context (LCC). Experiments on MDMD show that our method outperforms the best performing baseline by a large margin, i.e., 16.1%, 11.9%, 12.0% and 6.1% on precision, recall, F1, and Jaccard score, respectively. 1 Introduction Safety is an increasingly important aspect of artifi- cial intelligence development (Amodei et al., 2016; Roegiest et al., 2019; Sun et al., 2021). When it comes to dialogue agents, taking measures to avoid risks of generating undesirable and harmful responses may have a profound positive impact on the adoption of conversational technology (Xu et al., 2020). Research on safe dialogue agents in- volves aspects such as inaccurate information (Gun- son et al., 2021), fairness (Liu et al., 2020), and ∗ Corresponding author. unauthorized expertise (Sun et al., 2021). Malevo- lence is another key aspect (Zhang et al., 2021b,a), e.g., whether the dialogue utterance contains malev- olent content that is related to offensiveness (Dinan et al., 2019), toxicity (Gehman et al., 2020), ad hominem (Sheng et al., 2021), and toxicity agree- ment (Baheti et al., 2021), etc. There have been several studies targeting malev- olence detection (Roussinov and Robles-Flores, 2007; Saral et al., 2018; Zhang et al., 2021a,b). We build on the work of Zhang et al. (2021b) who introduce the malevolent dialogue response detec- tion and classification task, present a hierarchical malevolent dialogue taxonomy, create a labeled multi-turn dialogue data set, and apply state-of- the-art text classification methods to the task. One important limitation of their work is that they only explore single-label dialogue malevolence detec- tion (SDMD), i.e., they assume that each dialogue utterance corresponds to a single malevolence or non-malevolence label. However, some utterances have more than one label, e.g., in Figure 1, the utterance “f** people are …

Content truncated to 500 words