Federated Chinese Word Segmentation with Global Character Associations

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4306–4313 August 1–6, 2021. ©2021 Association for Computational Linguistics 4306 Federated Chinese Word Segmentation with Global Character Associations Yuanhe Tian♥∗, Guimin Chen♦∗, Han Qin♠∗, Yan Song♠♥† ♥University of Washington ♦QTrade ♠The Chinese University of Hong Kong (Shenzhen) ♥Shenzhen Research Institute of Big Data ♥yhtian@uw.edu ♦chenguimin@foxmail.com ♠hanqin@link.cuhk.edu.cn ♠songyan@cuhk.edu.cn Abstract Chinese word segmentation (CWS) is a fun- damental task for Chinese information pro- cessing, which always suffers from out-of- vocabulary word issues, especially when it is tested on data from different sources. Al- though one possible solution is to use more training data, in real applications, these data are stored at different locations and thus are in- visible and isolated among each other owing to the privacy or legal issues (e.g., clinical re- ports from different hospitals). To address this issue and benefit from extra data, we propose a neural model for CWS with federated learn- ing (FL) adopted to help CWS deal with data isolation, where a mechanism of global charac- ter associations is proposed to enhance FL to learn from different data sources. Experimen- tal results on a simulated environment with five nodes confirm the effectiveness of our ap- proach, where our approach outperforms dif- ferent baselines including some well-designed FL frameworks.1 1 Introduction Chinese word segmentation (CWS) is a prelimi- nary and vital task for natural language processing (NLP). This task aims to segment Chinese char- acter sequence into words and thus is generally performed as a sequence labeling task (Tseng et al., 2005; Levow, 2006; Song et al., 2009a; Sun and Xu, 2011; Song and Xia, 2012, 2013; Mansur et al., 2013). Although recent neural-based CWS sys- tems (Pei et al., 2014; Chen et al., 2017; Ma et al., 2018; Higashiyama et al., 2019; Qiu et al., 2019; Ke et al., 2020; Huang et al., 2020a; Tian et al., 2020e) have achieved very good performance on benchmark datasets, it is still an unsolved task (Fu *Equal contribution. †Corresponding author. 1The code and models involved in this paper are released at https://github.com/cuhksz-nlp/GCASeg. et al., 2020), because it is challenging to handle out-of-vocabulary words (OOV), especially in real applications where the test data may come from different sources. Although leveraging extra la- beled data from other sources or domains could alleviate this issue, in real applications, such data are always located in different nodes and thus are inaccessible to each other because of the privacy or legal concerns (e.g., clinical or financial reports from different hospitals or companies). To address the data isolation issue, feder- ated learning (FL) (Shokri and Shmatikov, 2015; Koneˇcn`y et al., 2016) is proposed and has shown its great promises for many machine learning tasks (Aono et al., 2017; Sheller et al., 2018; He et al., 2020). In many cases, data in different nodes are encrypted and aggregated to the centralized model, and they are invisible to each other during the train- ing stage. This property allows FL to be an es- sential technique for real …

Content truncated to 500 words