Neural Word Segmentation Learning for Chinese

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 409–420, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Neural Word Segmentation Learning for Chinese Deng Cai and Hai Zhao∗ Department of Computer Science and Engineering Key Lab of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering Shanghai Jiao Tong University, Shanghai, China thisisjcykcd@gmail.com, zhaohai@cs.sjtu.edu.cn Abstract Most previous approaches to Chinese word segmentation formalize this prob- lem as a character-based sequence label- ing task so that only contextual informa- tion within fixed sized local windows and simple interactions between adjacent tags can be captured. In this paper, we pro- pose a novel neural framework which thor- oughly eliminates context windows and can utilize complete segmentation history. Our model employs a gated combination neural network over characters to produce distributed representations of word candi- dates, which are then given to a long short- term memory (LSTM) language scoring model. Experiments on the benchmark datasets show that without the help of feature engineering as most existing ap- proaches, our models achieve competitive or better performances with previous state- of-the-art methods. 1 Introduction Most east Asian languages including Chinese are written without explicit word delimiters, therefore, word segmentation is a preliminary step for pro- cessing those languages. Since Xue (2003), most methods formalize the Chinese word segmentation (CWS) as a sequence labeling problem with char- acter position tags, which can be handled with su- ∗Corresponding author. This paper was partially sup- ported by Cai Yuanpei Program (CSC No. 201304490199 and No. 201304490171), National Natural Science Founda- tion of China (No. 61170114 and No. 61272248), National Basic Research Program of China (No. 2013CB329401), Major Basic Research Program of Shanghai Science and Technology Committee (No. 15JC1400103), Art and Sci- ence Interdisciplinary Funds of Shanghai Jiao Tong Univer- sity (No. 14JCRZ04), and Key Project of National Society Science Foundation of China (No. 15-ZDA041). pervised learning methods such as Maximum En- tropy (Berger et al., 1996; Low et al., 2005) and Conditional Random Fields (Lafferty et al., 2001; Peng et al., 2004; Zhao et al., 2006a). However, those methods heavily depend on the choice of handcrafted features. Recently, neural models have been widely used for NLP tasks for their ability to minimize the ef- fort in feature engineering. For the task of CWS, Zheng et al. (2013) adapted the general neural network architecture for sequence labeling pro- posed in (Collobert et al., 2011), and used char- acter embeddings as input to a two-layer network. Pei et al. (2014) improved upon (Zheng et al., 2013) by explicitly modeling the interactions be- tween local context and previous tag. Chen et al. (2015a) proposed a gated recursive neural network to model the feature combinations of context char- acters. Chen et al. (2015b) used an LSTM archi- tecture to capture potential long-distance depen- dencies, which alleviates the limitation of the size of context window but introduced another window for hidden states. Despite the differences, all these models are de- signed to solve CWS by assigning …

Content truncated to 500 words