Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task

Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task Pinzhen Chen Kenneth Heafield School of Informatics, University of Edinburgh {pinzhen.chen, kenneth.heafield}@ed.ac.uk Abstract Chinese word segmentation has entered the deep learning era which greatly reduces the hassle of feature engineering. Recently, some researchers attempted to treat it as character- level translation, which further simplified model designing, but there is a performance gap between the translation-based approach and other methods. This motivates our work, in which we apply the best practices from low- resource neural machine translation to super- vised Chinese segmentation. We examine a series of techniques including regularization, data augmentation, objective weighting, trans- fer learning, and ensembling. Compared to previous works, our low-resource translation- based method maintains the effortless model design, yet achieves the same result as state of the art in the constrained evaluation without using additional data. 1 Introduction Chinese text is written in characters as the smallest unit, and it has no explicit word boundary. There- fore, Chinese word segmentation (CWS) serves as upstream tokenization and disambiguation for Chinese language processing. The task is often viewed as sequence labelling, where each char- acter receives a label indicating its relative posi- tion in a segmented sequence (e.g. whether the character is at the word boundary). While tra- ditional machine learning methods have attained strong results, recent investigations focus on neu- ral networks given their rise in the entire NLP field. Distinctive to sequence labelling, Shi et al. (2017) first treat CWS as neural machine transla- tion (NMT). Nonetheless, Zhao et al. (2018) point out that without extra resources, all previous neu- ral methods are not yet comparable with the non- neural state of the art from Zhao and Kit (2008); the NMT practice is even behind. We note two advantages of treating the task as neural translation: 1) the entire input sentence is encoded before making any segmentation deci- sion; 2) such a model jointly trains character em- beddings with sequence modelling. Thus, we try to bridge the gap between the translation-based approach and state-of-the-art models, using low- resource techniques commonly seen in NMT. The translation-based method can be easy to adopt without the need for feature extraction and model modifictaion. Although NMT is known to be data hungry, our approach is able to achieve competi- tive results in the constrained evaluation scenario, where introducing extra data is forbidden. In specific, when benchmarked on the second CWS bakeoff (Emerson, 2005), our system reaches the top of the MSR leaderboard and achieves a strong result on the PKU dataset. 2 Related Work Chinese segmentation is traditionally tackled as sequence labelling, which predicts whether each input character should be split from neighbouring characters (Xue, 2003). Earlier approaches relied on conditional random fields or maximum entropy Markov models (Peng et al., 2004; Ng and Low, 2004). Zhao and Kit (2008) leveraged unsuper- vised features to attain state-of-the-art results in the data-constrained track. Recent research has shifted towards neural net- works: feed-forward, recurrent and convolutional (Zheng et al., 2013; Pei …

Content truncated to 500 words