Improving Chinese Word Segmentation with Wordhood Memory Networks

Improving Chinese Word Segmentation with Wordhood Memory Networks Yuanhe Tian♥∗, Yan Song♠† , Fei Xia♥, Tong Zhang♦, Yonggang Wang♠ ♥University of Washington, ♠Sinovation Ventures ♦The Hong Kong University of Science and Technology ♥{yhtian, fxia}@uw.edu ♠clksong@gmail.com ♦tongzhang@ust.hk ♠wangyonggang@chuangxin.com Abstract Contextual features always play an important role in Chinese word segmentation (CWS). Wordhood information, being one of the contextual features, is proved to be useful in many conventional character-based seg- menters. However, this feature receives less attention in recent neural models and it is also challenging to design a framework that can properly integrate wordhood information from different wordhood measures to existing neural frameworks. In this paper, we there- fore propose a neural framework, WMSEG, which uses memory networks to incorporate wordhood information with several popular encoder-decoder combinations for CWS. Ex- perimental results on five benchmark datasets indicate the memory mechanism successfully models wordhood information for neural seg- menters and helps WMSEG achieve state-of- the-art performance on all those datasets. Fur- ther experiments and analyses also demon- strate the robustness of our proposed frame- work with respect to different wordhood mea- sures and the efficiency of wordhood informa- tion in cross-domain experiments.1 1 Introduction Unlike most written languages in the world, the Chinese writing system does not use explicit de- limiters (e.g., white space) to separate words in written text. Therefore, Chinese word segmenta- tion (CWS) conventionally serves as the first step in Chinese language processing, especially for many downstream tasks such as text classification (Zeng et al., 2018), question answering (Liu et al., 2018), machine translation (Yang et al., 2018), etc. In the past two decades, the mainstream method- ology of CWS treated CWS as a character-based ∗Partially done as an intern at Sinovation Ventures. †Corresponding author. 1WMSEG (code and the best performing models) is re- leased at https://github.com/SVAIGBA/WMSeg. sequence labeling task (Tseng et al., 2005; Song et al., 2006; Sun and Xu, 2011; Pei et al., 2014; Chen et al., 2015; Zhang et al., 2016; Chen et al., 2017; Ma et al., 2018; Higashiyama et al., 2019; Qiu et al., 2019), where various studies were pro- posed to effectively extract contextual features to help better predicting segmentation labels for each character (Zhang et al., 2013; Zhou et al., 2017; Higashiyama et al., 2019). Among all the contex- tual features, the ones measuring wordhood for n-grams illustrate their helpfulness in many non- neural CWS models (Sun et al., 1998; Xue and Shen, 2003; Feng et al., 2004; Song and Xia, 2012). Later, following the track of the sequence label- ing methodology, recent approaches with neural networks are proved to be powerful in this task (Chen et al., 2015; Ma et al., 2018; Higashiyama et al., 2019). However, since neural networks (e.g., LSTM) is considered to be able to provide a good modeling of contextual dependencies, less attention is paid to the idea of explicitly leveraging word- hood information of n-grams in the context as what had previously been done in non-neural models. Al- though some studies sidestepped the idea by incor- …

Content truncated to 500 words