Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6829–6839 August 1–6, 2021. ©2021 Association for Computational Linguistics 6829 Privacy at Scale: Introducing the PrivaSeer Corpus of Web Privacy Policies Mukund Srinath Shomir Wilson C. Lee Giles College of Information Sciences and Technology Pennsylvania State University University Park, PA, USA {mukund, shomir, clg20}@psu.edu Abstract Organisations disclose their privacy practices by posting privacy policies on their websites. Even though internet users often care about their digital privacy, they usually do not read privacy policies, since understanding them re- quires a significant investment of time and ef- fort. Natural language processing has been used to create experimental tools to interpret privacy policies, but there has been a lack of large privacy policy corpora to facilitate the creation of large-scale semi-supervised and un- supervised models to interpret and simplify privacy policies. Thus, we present the Pri- vaSeer Corpus of 1,005,380 English language website privacy policies collected from the web. The number of unique websites repre- sented in PrivaSeer is about ten times larger than the next largest public collection of web privacy policies, and it surpasses the aggre- gate of unique websites represented in all other publicly available privacy policy corpora com- bined. We describe a corpus creation pipeline with stages that include a web crawler, lan- guage detection, document classification, du- plicate and near-duplicate removal, and con- tent extraction. We employ an unsupervised topic modelling approach to investigate the contents of policy documents in the corpus and discuss the distribution of topics in pri- vacy policies at web scale. We further inves- tigate the relationship between privacy policy domain PageRanks and text features of the pri- vacy policies. Finally, we use the corpus to pre- train PrivBERT, a transformer-based privacy policy language model, and obtain state of the art results on the data practice classification and question answering tasks. 1 Introduction A privacy policy is a legal document that an or- ganisation uses to disclose how they collect, ana- lyze, share, and protect users’ personal informa- tion. Legal jurisdictions around the world require organisations to make their privacy policies readily available to their users, and laws such as General Data Protection Regulation (GDPR) and Califor- nia Consumer Privacy Act (CCPA) place specific expectations upon privacy policies. However, al- though many internet users have concerns about their privacy (Madden, 2017), most fail to under- stand privacy policies (Meiselwitz, 2013). Studies show that privacy policies require a considerable in- vestment in time to read (Obar and Oeldorf-Hirsch, 2018) and estimate that it would require approxi- mately 200 hours to read all the privacy policies that an average person would come across every year (McDonald and Cranor, 2008). Natural language processing (NLP) provides an opportunity to automate the extraction of salient details from privacy policies, thereby reducing hu- man effort and enabling the creation of tools for internet users to understand and control their on- line privacy. Existing research has achieved …

Content truncated to 500 words