Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2814–2824 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 2814 Efficient-FedRec: Efficient Federated Learning Framework for Privacy-Preserving News Recommendation Jingwei Yi1, Fangzhao Wu2, Chuhan Wu3, Ruixuan Liu4, Guangzhong Sun1, Xing Xie2 1University of Science and Technology of China 2Microsoft Research Asia 3Tsinghua University 4Renmin University of China yjw1029@mail.ustc.edu.cn {wufangzhao,wuchuhan15}@gmail.com ruixuan.liu@ruc.edu.cn gzsun@ustc.edu.cn xingx@microsoft.com Abstract News recommendation is critical for person- alized news access. Most existing news rec- ommendation methods rely on centralized stor- age of users’ historical news click behav- ior data, which may lead to privacy con- cerns and hazards. Federated Learning is a privacy-preserving framework for multiple clients to collaboratively train models without sharing their private data. However, the com- putation and communication cost of directly learning many existing news recommendation models in a federated way are unacceptable for user clients. In this paper, we propose an efficient federated learning framework for privacy-preserving news recommendation. In- stead of training and communicating the whole model, we decompose the news recommenda- tion model into a large news model maintained in the server and a light-weight user model shared on both server and clients, where news representations and user model are communi- cated between server and clients. More specif- ically, the clients request the user model and news representations from the server, and send their locally computed gradients to the server for aggregation. The server updates its global user model with the aggregated gradients, and further updates its news model to infer updated news representations. Since the local gradi- ents may contain private information, we pro- pose a secure aggregation method to aggregate gradients in a privacy-preserving way. Experi- ments on two real-world datasets show that our method can reduce the computation and com- munication cost on clients while keep promis- ing model performance. 1 Introduction With the explosion of online information, the large quantities of news generated every day may over- whelm users and make them difficult to find the news they are interested in. To tackle this prob- lem, many news recommendation methods (An et al., 2019; Wang et al., 2020; Wu et al., 2019a; Qi et al., 2021c) have been proposed to display news according to users’ personalized interests. These methods are usually composed of two core mod- ules, i.e., user model and news model. The user model is used to learn user representations from user historical click behaviors. For example, Wang et al. (2018) use a candidate-aware attention net- work as the user model to help capture user inter- ests in candidate news. The news model is used to learn news representations from news content. For example, Wu et al. (2019c) apply multi-head self attention network to capture the interactions between words in news model. With the success of pre-trained language models (PLM) in NLP, a few PLM-empowered news recommendation methods have been proposed and achieve remarkable per- formance. For example, Wu et al. (2021b) apply pre-trained language models to enhance news mod- eling. However, …

Content truncated to 500 words