Style Obfuscation by Invariance

Proceedings of the 27th International Conference on Computational Linguistics, pages 984–996 Santa Fe, New Mexico, USA, August 20-26, 2018. 984 Style Obfuscation by Invariance Chris Emmery∗ Tilburg University University of Antwerp cmry@pm.me Enrique Manjavacas∗ University of Antwerp enrique.manjavacas @uantwerpen.be Grzegorz Chrupała Tilburg University g.chrupala@uvt.nl Abstract The task of obfuscating writing style using sequence models has previously been investigated under the framework of obfuscation-by-transfer, where the input text is explicitly rewritten in another style. A side effect of this framework are the frequent major alterations to the semantic content of the input. In this work, we propose obfuscation-by-invariance, and investigate to what extent models trained to be explicitly style-invariant preserve semantics. We evaluate our archi- tectures in parallel and non-parallel settings, and compare automatic and human evaluations on the obfuscated sentences. Our experiments show that the performance of a style classifier can be reduced to chance level, while the output is evaluated to be of equal quality to models applying style-transfer. Additionally, human evaluation indicates a trade-off between the level of obfusca- tion and the observed quality of the output in terms of meaning preservation and grammaticality. 1 Introduction The fact that writing style uniquely characterizes a person, and can be leveraged for automatic author identification (Holmes, 1998; Stamatatos et al., 2000), has been well-studied in the field of (compu- tational) stylometry (Neal et al., 2017). Similarly, work on author profiling (Koppel et al., 2002) has demonstrated that such stylometric features can be used to accurately infer an extensive set of personal information, such as age, gender, education, socio-economic status, and mental health issues (Eisenstein et al., 2011; Alowibdi et al., 2013; Volkova et al., 2014; Plank and Hovy, 2015; Volkova and Bachrach, 2016). Traditionally, these techniques relied on expensive human-labelled examples; however, more recent work has demonstrated near equal accuracy when only relying on self-reports as a distant super- vision signal (Beller et al., 2014; Emmery et al., 2017; Yates et al., 2017). While these efforts have been greatly beneficial to various research fields such as computational sociolinguistics (Daelemans, 2013), they potentially expose users of such media to attacks where this information can be abused unbeknownst to them. This is particularly harmful to individuals in a vulnerable position regarding race, political af- filiation, mental health, or any other personal information made explicitly unavailable. Adversarial stylometry, or style obfuscation, is one of the proposed methods aimed at protecting users against such attacks. Its objective is to rewrite an input text such that a classifier (the adversary) trained on detecting a particular variable (such as a demographic attribute) is fooled — effectively protecting this variable. The main challenge is to preserve the original meaning of the input, whilst hiding the act of obfuscation (Potthast et al., 2016). Recent work on automatic obfuscation (Shetty et al., 2017; Karadzhov et al., 2017) shows promising results in minimizing performance of the adversary; however, these models (also noted by the authors) struggle with correctly maintaining the semantic content of the input. To illustrate, while rewriting school …

Content truncated to 500 words