FIREBALL: A Dataset of Dungeons and Dragons Actual-Play with Structured Game State Information

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 364–376 July 9-14, 2023 ©2023 Association for Computational Linguistics Rule By Example: Harnessing Logical Rules for Explainable Hate Speech Detection Christopher Clarke˚: Matthew Hall; Gaurav Mittal; Ye Yu; Sandra Sajeev; Jason Mars: Mei Chen; :University of Michigan, Ann Arbor, MI ;Microsoft, Redmond, WA {csclarke, profmars}@umich.edu {mathall, gaurav.mittal, yu.ye, ssajeev, mei.chen}@microsoft.com Abstract Classic approaches to content moderation typi- cally apply a rule-based heuristic approach to flag content. While rules are easily customiz- able and intuitive for humans to interpret, they are inherently fragile and lack the flexibility or robustness needed to moderate the vast amount of undesirable content found online today. Re- cent advances in deep learning have demon- strated the promise of using highly effective deep neural models to overcome these chal- lenges. However, despite the improved per- formance, these data-driven models lack trans- parency and explainability, often leading to mis- trust from everyday users and a lack of adoption by many platforms. In this paper, we present Rule By Example (RBE): a novel exemplar- based contrastive learning approach for learn- ing from logical rules for the task of textual content moderation. RBE is capable of pro- viding rule-grounded predictions, allowing for more explainable and customizable predictions compared to typical deep learning-based ap- proaches. We demonstrate that our approach is capable of learning rich rule embedding rep- resentations using only a few data examples. Experimental results on 3 popular hate speech classification datasets show that RBE is able to outperform state-of-the-art deep learning clas- sifiers as well as the use of rules in both su- pervised and unsupervised settings while pro- viding explainable model predictions via rule- grounding. 1 Introduction Content moderation is a major challenge con- fronting the safety of online social platforms such as Facebook, Twitter, YouTube, Twitch, etc. (Vaidya et al., 2021). Major technology corpora- tions are increasingly allocating valuable resources towards the development of automated systems for *This work was done as Christopher’s internship project at Microsoft. Figure 1: Generalization problem of rules. Logical rules, while easy to explain, are inherently fragile to the nuances of natural language. the detection and moderation of harmful content in addition to hiring and training expert human mod- erators to combat the growing menace of negativity and toxicity online (Wagner and Bloomberg, 2021; Liu et al., 2022). Despite the popularity of deep learning ap- proaches, many practical solutions used in products today are comprised of rule-based techniques based on expertly curated signals such as block lists, key phrases, and regular expressions (Gillespie, 2018; Zhang, 2019; Dada et al., 2019). Such methods are widely used due to their transparency, ease of customization, and interpretability. However, they have the disadvantage of being difficult to maintain and scale, in addition to being inherently fragile and noisy (Zhang, 2019; Davidson et al., 2017; Lee, 2022; Lai et al., 2022). Figure 1 shows an example where logical rules, while explainable in nature, face the problem of being inflexible …

Content truncated to 500 words