Peeking inside the black box: A Commonsense-aware Generative Framework for Explainable Complaint Detection

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 7333–7347 July 9-14, 2023 ©2023 Association for Computational Linguistics Peeking inside the black box: A Commonsense-Aware Generative Framework for Explainable Complaint Detection Apoorva Singh1⋆, Raghav Jain1⋆, Prince Jha1 and Sriparna Saha1 1Department of Computer Science and Engineering, IIT Patna, India {apoorva_1921cs19,sriparna}@iitp.ac.in, {raghavjain106, jhapks1999}@gmail.com ∗ Abstract Complaining is an illocutionary act in which the speaker communicates his/her dissatisfac- tion with a set of circumstances and holds the hearer (the complainee) answerable, directly or indirectly. Considering breakthroughs in ma- chine learning approaches, the complaint detec- tion task has piqued the interest of the natural language processing (NLP) community. Most of the earlier studies failed to justify their find- ings, necessitating the adoption of interpretable models that can explain the model’s output in real-time. We introduce an explainable com- plaint dataset, X-CI, the first benchmark dataset for explainable complaint detection. Each in- stance in the X-CI dataset is annotated with five labels: complaint label, emotion label, po- larity label, complaint severity level, and ra- tionale (explainability), i.e., the causal span explaining the reason for the complaint/non- complaint label. We address the task of ex- plainable complaint detection and propose a commonsense-aware unified generative frame- work by reframing the multitask problem as a text-to-text generation task. Our framework can predict the complaint cause, severity level, emotion, and polarity of the text in addition to detecting whether it is a complaint or not. We further establish the advantages of our pro- posed model on various evaluation metrics over the state-of-the-art models and other baselines when applied to the X-CI dataset in both full and few-shot settings1. 1 Introduction Complaining is an expression of negative emotions communicated due to a discrepancy between reality and expectations (Olshtain and Weinbach, 1985). In pragmatics theory, Trosborg (2011) proposed four primary complaint severity levels: (a) no ex- plicit reproach, (b) disapproval, (c) accusation, and ∗* The first two authors contributed equally to this work and are jointly the first authors. 1The dataset and code are available at https://github. com/appy1608/ACL2023 (d) blame. Recent studies on complaint detection have mainly dealt with automatically identifying binary complaints and the associated severity lev- els from social-media data (Preotiuc-Pietro et al., 2019; Jin and Aletras, 2021). However, these stud- ies primarily focused on improving complaint de- tection performance with the help of various mod- els without providing any perspective or evaluation of the outcome’s explainability. Since the intro- duction of explainable artificial intelligence (AI) (Gunning et al., 2019), providing interpretation for any AI algorithm’s decision has become essential. Therefore, rather than enhancing performance by increasing computational burden, there is a push to construct trustworthy and transparent interpretable models. As sentient beings, we use our commonsense to establish connections between what is explicitly said and inferred. As a result, we believe that adding external knowledge or commonsense knowl- edge (Sabour et al., 2021a) to complaint detection systems can help them better grasp the user’s cir- cumstances and concerns, resulting …

Content truncated to 500 words