Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2388–2402 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2388 Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling Chris Emmery CSAI, Tilburg University CLiPS, University of Antwerp cmry@pm.me ´Akos K´ad´ar Borealis AI akos.kadar@ borealisai.com Grzegorz Chrupała CSAI, Tilburg University g.a.chrupala@uvt.nl Abstract Written language contains stylistic cues that can be exploited to automatically infer a vari- ety of potentially sensitive author information. Adversarial stylometry intends to attack such models by rewriting an author’s text. Our re- search proposes several components to facili- tate deployment of these adversarial attacks in the wild, where neither data nor target mod- els are accessible. We introduce a transformer- based extension of a lexical replacement at- tack, and show it achieves high transferabil- ity when trained on a weakly labeled corpus— decreasing target model performance below chance. While not completely inconspicuous, our more successful attacks also prove notably less detectable by humans. Our framework therefore provides a promising direction for fu- ture privacy-preserving adversarial attacks. 1 Introduction The widespread use of machine learning on con- sumer devices and its application to their data has sparked investigation of security and privacy re- searchers alike in correctly handling sensitive infor- mation (Edwards and Storkey, 2016; Abadi et al., 2016b). Natural Language Processing (NLP) is no exception (Fernandes et al., 2019; Li et al., 2018); written text can contain a plethora of author information—either consciously shared or infer- able through stylometric analysis (Rao et al., 2000; Adams, 2006). This characteristic is fundamental to author profiling (Koppel et al., 2002), and while the field’s main interest pertains to the study of sociolinguistic and stylometric features that under- pin our language use (Daelemans, 2013), herein simultaneously lie its dual-use problems. Author profiling can, often with high accuracy, infer an ex- tensive set of (sensitive) personal information, such as age, gender, education, socio-economic status, and mental health issues (Eisenstein et al., 2011; Alowibdi et al., 2013; Volkova et al., 2014; Plank and Hovy, 2015; Volkova and Bachrach, 2016). It therefore potentially exposes anyone sharing writ- ten online content to unauthorized information col- lection through their writing style. This can prove particularly harmful to individuals in a vulnerable position regarding e.g., race, political affiliation, or mental health. Privacy-preserving defenses against such infer- ences can be found in the field of adversarial1 sty- lometry. Our research2 concerns the obfuscation subtask, where the aim is to rewrite an input text such that the style changes, and stylometric predic- tions fail. It is part of a growing body of research into adversarial attacks on NLP (Smith, 2012), which various modern models have proven vulnera- ble to; e.g., in neural machine translation (Ebrahimi et al., 2018), summarization (Cheng et al., 2020), and text classification (Liang et al., 2018). Adversarial attacks on NLP are predominantly aimed at demonstrating vulnerabilities in existing algorithms or models, such that they might be fixed, or explicitly improved through adversarial training. …

Content truncated to 500 words