Adversarial Training for Satire Detection: Controlling for Confounding Variables

Proceedings of NAACL-HLT 2019, pages 660–665 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 660 Adversarial Training for Satire Detection: Controlling for Confounding Variables Robert McHardy1, Heike Adel1,2∗ and Roman Klinger1 1 Institut f¨ur Maschinelle Sprachverarbeitung, University of Stuttgart, Germany 2 Bosch Center for Artificial Intelligence, Renningen, Germany {firstname.lastname}@ims.uni-stuttgart.de Abstract The automatic detection of satire vs. regu- lar news is relevant for downstream applica- tions (for instance, knowledge base popula- tion) and to improve the understanding of lin- guistic characteristics of satire. Recent ap- proaches build upon corpora which have been labeled automatically based on article sources. We hypothesize that this encourages the mod- els to learn characteristics for different pub- lication sources (e.g., “The Onion” vs. “The Guardian”) rather than characteristics of satire, leading to poor generalization performance to unseen publication sources. We therefore pro- pose a novel model for satire detection with an adversarial component to control for the con- founding variable of publication source. On a large novel data set collected from German news (which we make available to the research community), we observe comparable satire classification performance and, as desired, a considerable drop in publication classification performance with adversarial training. Our analysis shows that the adversarial component is crucial for the model to learn to pay atten- tion to linguistic properties of satire. 1 Introduction Satire is a form of art used to criticize in an en- tertaining manner (cf. Sulzer, 1771, p. 995ff.). It makes use of different stylistic devices, e.g., humor, irony, sarcasm, exaggerations, parody or caricature (Knoche, 1982; Colletta, 2009). The occurrence of harsh, offensive or banal and funny words is typical (Golbert, 1962; Brummack, 1971). Satirical news are written with the aim of mim- icking regular news in diction. In contrast to misin- formation and disinformation (Thorne and Vlachos, 2018), it does not have the intention of fooling the readers into actually believing something wrong in order to manipulate their opinion. * Work was done at University of Stuttgart. The task of satire detection is to automatically distinguish satirical news from regular news. This is relevant, for instance, for downstream applica- tions, such that satirical articles can be ignored in knowledge base population. Solving this problem computationally is challenging. Even human read- ers are sometimes not able to precisely recognize satire (Allcott and Gentzkow, 2017). Thus, an auto- matic system for satire detection is both relevant for downstream applications and could help humans to better understand the characteristics of satire. Previous work mostly builds on top of corpora of news articles which have been labeled automat- ically based on the publication source (e.g., “The New York Times” articles would be labeled as reg- ular while “The Onion” articles as satire1). We hy- pothesize that such distant labeling approach leads to the model mostly representing characteristics of the publishers instead of actual satire. This has two main issues: First, interpretation of the model to obtain a better understanding of concepts of satire would be misleading, and second, generalization …

Content truncated to 500 words