Personal Information Leakage Detection in Conversations

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6567–6580, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 6567 Personal Information Leakage Detection in Conversations Qiongkai Xu1,2, Lizhen Qu3∗, Zeyu Gao1, Gholamreza Haffari3 1The Australian National University, Canberra, ACT, Australia 2Data61 CSIRO, Canberra, ACT, Australia 3Monash University, Clayton, VIC, Australia {Qiongkai.Xu,Zeyu.Gao}@anu.edu.au {Lizhen.Qu,Gholamreza.Haffari}@monash.edu Abstract The global market size of conversational assis- tants (chatbots) is expected to grow to USD 9.4 billion by 2024, according to Marketsand- Markets. Despite the wide use of chatbots, leakage of personal information through chat- bots poses serious privacy concerns for their users. In this work, we propose to protect personal information by warning users of de- tected suspicious sentences generated by con- versational assistants. The detection task is for- mulated as an alignment optimization problem and a new dataset PERSONA-LEAKAGE is collected for evaluation. In this paper, we pro- pose two novel constrained alignment models, which consistently outperform baseline meth- ods on PERSONA-LEAKAGE1. Moreover, we conduct analysis on the behavior of recently proposed personalized chit-chat dialogue sys- tems. The empirical results show that those systems suffer more from personal informa- tion disclosure than the widely used Seq2Seq model and the language model. In those cases, a significant number of information leaking ut- terances can be detected by our models with high precision. 1 Introduction According to Opus Research2, 4.5 billion dollars will be invested in conversational assistants (chat- bots) by 2021. Among diverse types of chatbots, Google Duplex, first introduced at Google I/O 2018, represents the kind of AI personal assistants (PAs) that act on behalf of people to perform sim- ple tasks, such as making reservations at restaurants and hair salons. In order to successfully complete those tasks, PAs are granted the access to personal information (PI) of their owners, such as number of ∗*Corresponding author 1The dataset and our model implementation is available at https://github.com/xuqiongkai/PILD. 2https://www.opus.global/media/44137/ opus-q3-2018-report-eng.pdf Figure 1: Given utterances (U) and personal informa- tion descriptions (P) from a conversational assistant (a), PILD module (b) detects risky utterances with cor- responding personal information and sends a warning (red arrow) to an authorized user (c). The authorized user manually approve or reject the utterances. Then, only the approved utterances (green arrow) are sent to interlocutors (d) who could be authorized or malicious. children, working hours, home address, and vaca- tion plans. Thus, these PAs pose privacy concerns when they communicate with real-life people, or other bots in natural language. Another major source of personal information leakage is online social networks, which store a huge amount of possibly sensitive information on users and their interactions (Zhang et al., 2010). However, a recent study shows that none of the pop- ular social network platforms (Facebook, Wechat, Google+, etc) have developed a perfectly non-leaky privacy protection mechanism (Yu et al., 2018). In addition, internet users (including a vast number of children and teenagers) often show a phenomenon called privacy paradox, which states that even users with high level of privacy concerns do not …

Content truncated to 500 words