From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 15162–15180 July 9-14, 2023 ©2023 Association for Computational Linguistics From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models Julia Mendelsohn♢ ∗ Ronan Le Bras♣ Yejin Choi♠♣ Maarten Sap♡♣ ♢University of Michigan School of Information ♣Allen Institute for AI ♠Paul G. Allen School of Computer Science & Engineering, University of Washington ♡Language Technologies Institute, Carnegie Mellon University � juliame@umich.edu � dogwhistles.allen.ai Abstract Warning: content in this paper may be upset- ting or offensive to some readers. Dogwhistles are coded expressions that simul- taneously convey one meaning to a broad audi- ence and a second one, often hateful or provoca- tive, to a narrow in-group; they are deployed to evade both political repercussions and algo- rithmic content moderation. For example, in the sentence “we need to end the cosmopolitan experiment,” the word “cosmopolitan” likely means “worldly” to many, but secretly means “Jewish” to a select few. We present the first large-scale computational investigation of dog- whistles. We develop a typology of dogwhis- tles, curate the largest-to-date glossary of over 300 dogwhistles with rich contextual informa- tion and examples, and analyze their usage in historical U.S. politicians’ speeches. We then assess whether a large language model (GPT- 3) can identify dogwhistles and their mean- ings, and find that GPT-3’s performance varies widely across types of dogwhistles and targeted groups. Finally, we show that harmful content containing dogwhistles avoids toxicity detec- tion, highlighting online risks of such coded language. This work sheds light on the theo- retical and applied importance of dogwhistles in both NLP and computational social science, and provides resources for future research in modeling dogwhistles and mitigating their on- line harms. 1 Introduction The cosmopolitan elite look down on the com- mon affections that once bound this nation to- gether: things like place and national feeling and religious faith... The cosmopolitan agenda has driven both Left and Right...It’s time we ended the cosmopolitan experiment and recovered the promise of the republic. –Josh Hawley (R-MO), 2019 ∗Work done while interning at the Allen Institute for AI. Figure 1: Schematic of how dogwhistles work, based on Henderson and McCready (2018) with the exam- ple of cosmopolitan. First, a speaker simultaneously communicates the dogwhistle message and their per- sona (identity). The in-group recovers both the message content and speaker persona, enabling them to arrive at the coded meaning (e.g. Jewish). The out-group only recognizes the message’s content and thus interprets it literally. This literal meaning also provides the speaker with plausible deniability; if confronted, the speaker can claim that they solely intended the literal meaning. We have got this tailspin of culture, in our inner cities in particular, of men not working and just generations of men not even thinking about working or learning to value the culture of work. –Paul Ryan (R-WI), 2014 Cosmopolitan and inner city are examples of dogwhistles, expressions that “send one message to an out-group and a second (often taboo, contro- …

Content truncated to 500 words