Interactive Text Ranking with Bayesian Optimization: A Case Study on Community QA and Summarization

Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System? Sorami Hisamoto∗ Works Applications s@89.io Matt Post Kevin Duh Johns Hopkins University {post,kevinduh}@cs.jhu.edu Abstract Data privacy is an important issue for ‘‘machine learning as a service’’ provid- ers. We focus on the problem of mem- bership inference attacks: Given a data sample and black-box access to a model’s API, determine whether the sample ex- isted in the model’s training data. Our contribution is an investigation of this problem in the context of sequence-to- sequence models, which are important in applications such as machine translation and video captioning. We define the mem- bership inference problem for sequence generation, provide an open dataset based on state-of-the-artmachinetranslation mod- els, and report initial results on whether thesemodelsleakprivate informationagainst several kinds of membership inference attacks. 1 Motivation There are many situations in which private entities are worried about the privacy of their data. For example, many companies provide black-box training services where users are able to upload their data and have customized models built for them, without requiring machine learning expertise. A common concern in these ‘‘machine learning as a service’’ offerings is that the up- loaded data be visible only to the client that owns it. Currently, these entities are in the position of having to trust that service providers abide by the terms of their agreements. Although trust is an important component in relationships of all kinds, it has its limitations. In particular, it falls short of a well-known security maxim, originating in a Russian proverb that translates as, Trust, but ∗Work done while visiting Johns Hopkins University. verify.1 Ideally, customers would be able to verify that their private data was not being slurped up by the serving company, whether by design or accident. This problem has been formalized as the mem- bership inference problem, first introduced by Shokri et al. (2017) and defined as: ‘‘Given a machine learning model and a record, determine whether this record was used as part of the model’s training dataset or not.’’ The problem can be tackled in an adversarial framework: The attacker is interested in answering this question with high accuracy, whereas the defender would like this question to be unanswerable (see Figure 1). Since then, researchers have proposed many ways to attack and defend the privacy of various types of models. However, the work so far has only focused on standard classification problems, where the output space of the model is a fixed set of labels. Inthis paper, we propose to investigate member- ship inference for sequence generation problems, where the output space can be viewed as a chained sequence of classifications. Prime examples of sequence generation includes machine translation and text summarization: In these problems, the output is a sequence of words whose length is un- determined a priori. Other examples include speech synthesis and video caption generation. Sequence generation problems are more complex than clas- sification problems, and it is unclear whether the methods and results developed for membership …

Content truncated to 500 words