Social Commonsense for Explanation and Cultural Bias Discovery

Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3745–3760 May 2-6, 2023 ©2023 Association for Computational Linguistics Social Commonsense for Explanation and Cultural Bias Discovery Lisa Bauer Hanna Tischer UNC Chapel HillMohit Bansal {lbauer6, hannalt, mbansal}@cs.unc.edu Abstract Social commonsense contains many human bi- ases due to social and cultural influence (Sap et al., 2020; Emelin et al., 2020). We focus on identifying cultural biases in data, specif- ically causal assumptions and commonsense implications, that strongly influence model de- cisions for a variety of tasks designed for so- cial impact. This enables us to examine data for bias by making explicit the causal (if-then, inferential) relations in social commonsense knowledge used for decision making, further- ing interpretable commonsense reasoning from a dataset perspective. We apply our methods on 2 social tasks: emotion detection and per- ceived value detection. We identify influen- tial social commonsense knowledge to explain model behavior in the following ways. First, we augment large-scale language models with social knowledge and show improvements for the tasks, indicating the implicit assumptions a model requires to be successful on each dataset. Second, we identify influential events in the datasets by using social knowledge to cluster data and demonstrate the influence that these events have on model behavior via leave-K-out experiments. This allows us to gain a dataset- level understanding of the events and causal commonsense relationships that strongly influ- ence predictions. We then analyze these rela- tionships to detect influential cultural bias in each dataset. Finally, we use our influential event identification for detecting mislabeled ex- amples and improve training and performance through their removal. We support our findings with manual analysis. 1 Introduction Social commonsense knowledge helps humans ma- neuver through everyday life, aiding in situations that may require social nuance or cultural knowl- edge. Commonsense knowledge acquisition has been an important goal in NLP (Levesque et al., 2012; Davis and Marcus, 2015; Talmor et al., 2019) and in the past few years there has been a surge of research focusing specifically on improving so- cial commonsense understanding for neural models (Sap et al., 2019a,b; Hwang et al., 2020; Forbes et al., 2020). However, social commonsense may contain many human biases due to social and cul- tural influence (Sap et al., 2020; Emelin et al., 2020). We aim to discover influential cultural bi- ases in social applications datasets via the social and causal commonsense knowledge (Roemmele et al., 2011; Luo et al., 2016; Ponti et al., 2020) present in each dataset, to identify bias in cause- effect commonsense relationships. We define cul- tural bias as a positive or negative cultural atti- tude toward a social structure (see Section 3.1) and define causal commonsense following Sap et al. (2019a), as if-then, inferential relations. Specifi- cally, we are interested in exploring biases towards the following social structures: religion, economy, family, government, education and technology. To the best of our knowledge, we are the first to empir- ically discover influential cultural biases in social tasks, …

Content truncated to 500 words