Through the Looking Glass: Learning to Attribute Synthetic Text Generated by Language Models

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 1811–1822 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 1811 Through the Looking Glass: Learning to Attribute Synthetic Text Generated by Language Models Shaoor Munir† Brishna Batool† Zubair Shafiq‡ Padmini Srinivasan∗ Fareed Zaffar† †Lahore University of Management Sciences, Pakistan shaoor.munir@lums.edu.pk ‡University of California at Davis, USA zubair@ucdavis.edu ∗University of Iowa, USA padmini-srinivasan@uiowa.edu Abstract Given the potential misuse of recent advances in synthetic text generation by language mod- els (LMs), it is important to have the capacity to attribute authorship of synthetic text. While stylometric organic (i.e., human written) au- thorship attribution has been quite successful, it is unclear whether similar approaches can be used to attribute a synthetic text to its source LM. We address this question with the key insight that synthetic texts carry subtle distin- guishing marks inherited from their source LM and that these marks can be leveraged by ma- chine learning (ML) algorithms for attribution. We propose and test several ML-based attribu- tion methods. Our best attributor built using a fine-tuned version of XLNet (XLNet-FT) con- sistently achieves excellent accuracy scores (91% to near perfect 98%) in terms of attribut- ing the parent pre-trained LM behind a syn- thetic text. Our experiments show promising results across a range of experiments where the synthetic text may be generated using pre- trained LMs, fine-tuned LMs, or by varying text generation parameters. 1 Introduction Recent advancements in natural language process- ing have enabled synthetic text generation that is often of comparable quality to the organic text (Ip- polito et al., 2020; Radford et al., 2019; Zellers et al., 2019; Gehrmann et al., 2019). This ca- pability has the potential to be misused by mali- cious actors to launch misinformation, spam, and phishing campaigns (Solaiman et al., 2019; Brown et al., 2020). To prevent potential misuse, prior research has shown considerable success in build- ing machine learning (ML) algorithms that detect (Zellers et al., 2019) or assist humans in detecting (Gehrmann et al., 2019) synthetic text. While prior research has shown promise in distin- guishing between synthetic and organic text, very little has been done on attributing the authorship of the language model (LM) generating the synthetic text (Pan et al., 2020). It is important to be able to track the provenance of synthetic text to the source LM. This can be useful in identifying perpetrators of potential misuse and the unauthorized use of an LM (e.g., in case it is stolen through sophisticated model inversion attacks (Fredrikson et al., 2015) or outright security breaches). It is particularly challenging to attribute the au- thorship of the synthetic texts because of the variety and number of available LMs and configurations. While there are only a handful of public pre-trained LMs, it is common to further fine-tune them before using them to generate synthetic text (Devlin et al., 2019; Sanh et al., 2020). Fine-tuning can signifi- cantly impact the characteristics of the generated text …

Content truncated to 500 words