A Girl Has A Name: Detecting Authorship Obfuscation

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2235–2245 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 2235 A Girl Has A Name: Detecting Authorship Obfuscation Asad Mahmood Zubair Shafiq Padmini Srinivasan The University of Iowa {asad-mahmood,zubair-shafiq,padmini-srinivasan}@uiowa.edu Abstract Authorship attribution aims to identify the au- thor of a text based on the stylometric analy- sis. Authorship obfuscation, on the other hand, aims to protect against authorship attribution by modifying a text’s style. In this paper, we evaluate the stealthiness of state-of-the-art au- thorship obfuscation methods under an adver- sarial threat model. An obfuscator is stealthy to the extent an adversary finds it challenging to detect whether or not a text modified by the obfuscator is obfuscated – a decision that is key to the adversary interested in authorship attribution. We show that the existing author- ship obfuscation methods are not stealthy as their obfuscated texts can be identified with an average F1 score of 0.87. The reason for the lack of stealthiness is that these obfusca- tors degrade text smoothness, as ascertained by neural language models, in a detectable manner. Our results highlight the need to de- velop stealthy authorship obfuscation methods that can better protect the identity of an author seeking anonymity. 1 Introduction Authorship attribution aims to identify the author of a text using stylometric techniques designed to capitalize on differences in the writing style of different authors. Owing to recent advances in machine learning, authorship attribution methods can now identify authors with impressive accuracy (Abbasi and Chen, 2008) even in challenging set- tings such as cross-domain (Overdorf and Green- stadt, 2016) and at a large-scale (Narayanan et al., 2012; Ruder et al., 2016). Such powerful author- ship attribution methods pose a threat to privacy- conscious users such as journalists and activists who may wish to publish anonymously (Times, 2018; Anonymous, 2018). Authorship obfuscation, a protective counter- measure, aims to evade authorship attribution by obfuscating the writing style in a text. Since it is challenging to accomplish this manually, re- searchers have developed automated authorship obfuscation methods that can evade attribution while preserving semantics (PAN, 2018). How- ever, a key limitation of prior work is that author- ship obfuscation methods do not consider the ad- versarial threat model where the adversary is “ob- fuscation aware” (Karadzhov et al., 2017; Potthast et al., 2018; Mahmood et al., 2019). Thus, in addi- tion to evading attribution and preserving seman- tics, it is important that authorship obfuscation methods are “stealthy” – i.e., they need to hide the fact that text was obfuscated from the adversary. In this paper, we investigate the stealthiness of state-of-the-art authorship obfuscation meth- ods. Our intuition is that the application of au- thorship obfuscation results in subtle differences in text smoothness (as compared to human writ- ing) that can be exploited for obfuscation detec- tion. To capitalize on this intuition, we use off-the- shelf pre-trained neural language models such as BERT and GPT-2 to extract text smoothness fea- tures in …

Content truncated to 500 words