FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings

Findings of the Association for Computational Linguistics: EACL 2023, pages 869–882 May 2-6, 2023 ©2023 Association for Computational Linguistics FedPerC: Federated Learning for Language Generation with Personal and Context Preference Embeddings Andrew Silva∗ Georgia Institute of Technology School of Interactive Computing Atlanta, GA andrew.silva@gatech.edu Pradyumna Tambwekar∗ Georgia Institute of Technology School of Interactive Computing Atlanta, GA pradyumna.tambwekar@gatech.edu Matthew Gombolay Georgia Institute of Technology School of Interactive Computing Atlanta, GA matthew.gombolay@cc.gatech.edu Abstract Federated learning is a training paradigm that learns from multiple distributed users without aggregating data on a centralized server, promising the ability to deploy machine- learning to a diverse population of users with- out first collecting large, labeled datasets. As federated learning involves averaging gradient updates across a decentralized population, there is a growing need for personalization of fed- erated learning systems (i.e. conversational agents must personalize to individual users and the context of an interaction). In this work, we propose a new direction for personaliza- tion research within federated learning, lever- aging both personal embeddings and shared context embeddings. We also present an ap- proach to predict these “preference” embed- dings, enabling personalization without back- propagation. Compared to state-of-the-art per- sonalization baselines, our approach achieves a 50% improvement in test-time perplexity using 0.001% of the memory required by baseline approaches, and achieving greater sample- and compute-efficiency. 1 Introduction As conversational agents and dialog systems are deployed to real-world scenarios, these systems require data-efficient personalization paradigms such that language systems such as conversational agents can be effectively adapted on-device. The benefits of on-device optimization are two-fold; (1) Swift adaptation of model-behavior based on human-interactions (Dudy et al., 2021), (2) Privacy protection by means of retaining all data related ∗ The authors contribute equally to this paper. to the user on-device (Li et al., 2020b). One of the prevailing paradigms for learning from and en- gaging with end-users is federated learning. Feder- ated learning is an inherently decentralized learning paradigm that assumes no access to a large labeled dataset and instead leverages averaged parameter updates across all users of the system (McMahan et al., 2017). Such averaged updates invariably dilute individual preferences or deviations from the mean, resulting in a model that works well for the average user while failing to appropriately cap- ture under-represented preferences or sub-groups within the data. In this work, we present a novel ap- proach (FedPerC) to personalizing federated learn- ing with personal and context embeddings (collec- tively called “preference embeddings”), adapting more efficiently and effectively than prior work with respect to both data and compute on-device. We leverage the insight that a client’s data distri- bution is informed by both individual preferences and additional contextual information. For exam- ple, while each user may have their own individual style, there may be more general population-wide trends that inform the style of personalized predic- tions (e.g., dialogue assistants helping patients with cognitive disorders, whereby agents can personal- ize to individual patients and broader condition- wide trends). While individual preferences may be unique to each client (e.g. …

Content truncated to 500 words