Neural Linguistic Steganography

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 1210–1215, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 1210 Neural Linguistic Steganography Zachary M. Ziegler∗ Yuntian Deng∗ Alexander M. Rush School of Engineering and Applied Sciences Harvard University, Cambridge, MA {zziegler@g, dengyuntian@g, srush@seas}.harvard.edu Abstract Whereas traditional cryptography encrypts a secret message into an unintelligible form, steganography conceals that communication is taking place by encoding a secret message into a cover signal. Language is a particu- larly pragmatic cover signal due to its benign occurrence and independence from any one medium. Traditionally, linguistic steganogra- phy systems encode secret messages in exist- ing text via synonym substitution or word or- der rearrangements. Advances in neural lan- guage models enable previously impractical generation-based techniques. We propose a steganography technique based on arithmetic coding with large-scale neural language mod- els. We find that our approach can generate realistic looking cover sentences as evaluated by humans, while at the same time preserving security by matching the cover message distri- bution with the language model distribution. 1 Introduction Cryptography is central to modern communica- tion, but while it effectively conceals the content of a message it reveals that meaningful communi- cation is taking place. Steganography answers an alternative question: how to conceal a message in some cover signal (an image, text etc.) such that an eavesdropper is not even aware any meaningful communication has taken place (Amin et al., 2003; Shirali-Shahreza and Shirali-Shahreza, 2007; Ku- mar and Pooja, 2010)? Different from cryptogra- phy, in steganography security is derived from the inability to detect that a message exists within the cover signal, rather than the inability of an eaves- dropper to determine the content of the message (Westfeld and Pfitzmann, 1999). Natural language is an especially useful cover signal for steganography because it is prevalent ∗Equal contribution. and innocuous in everyday life. Furthermore, un- like images or audio in which encoded information depends on exact pixel values, linguistic steganog- raphy (hiding information in choices of words) is untethered from any one medium. For example, the sender could encode a message on a computer, read off the cover text in person, and then the re- ceiver could separately enter the cover text into her decoder and retrieve the message. Linguistic steganography methods can be clas- sified as either edit-based or generation-based (Bennett, 2004). In edit-based methods, such as synonym substitution, the content of the cover text is selected by a human and slightly modified to encode information (Xiang et al., 2017; Hefei, 2009; Xiang et al., 2018). In generative methods an entire block of text is generated while encod- ing the message reversibly in the choice of to- kens. Traditionally, most practical stenography systems are edit-based. These methods only en- code a small amount of information (for example, 2 bits per tweet (Wilson and Ker, 2016)), whereas generation-based systems can encode an order of magnitude more information (Fang et al., 2017). …

Content truncated to 500 words