Long Short-Term Memory Neural Networks for Chinese Word Segmentation

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1197–1206, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Long Short-Term Memory Neural Networks for Chinese Word Segmentation Xinchi Chen, Xipeng Qiu∗, Chenxi Zhu, Pengfei Liu, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {xinchichen13,xpqiu,czhu13,pfliu14,xjhuang}@fudan.edu.cn Abstract Currently most of state-of-the-art meth- ods for Chinese word segmentation are based on supervised learning, whose fea- tures are mostly extracted from a local con- text. These methods cannot utilize the long distance information which is also crucial for word segmentation. In this paper, we propose a novel neural network model for Chinese word segmentation, which adopts the long short-term memory (LSTM) neu- ral network to keep the previous impor- tant information in memory cell and avoids the limit of window size of local context. Experiments on PKU, MSRA and CTB6 benchmark datasets show that our model outperforms the previous neural network models and state-of-the-art methods. 1 Introduction Word segmentation is a fundamental task for Chi- nese language processing. In recent years, Chi- nese word segmentation (CWS) has undergone great development. The popular method is to re- gard word segmentation task as a sequence label- ing problem (Xue, 2003; Peng et al., 2004). The goal of sequence labeling is to assign labels to all elements in a sequence, which can be handled with supervised learning algorithms such as Maximum Entropy (ME) (Berger et al., 1996) and Condi- tional Random Fields (CRF) (Lafferty et al., 2001). However, the ability of these models is restricted by the design of features, and the number of fea- tures could be so large that the result models are too large for practical use and prone to overfit on training corpus. Recently, neural network models have increas- ingly used for NLP tasks for their ability to min- imize the effort in feature engineering (Collobert ∗Corresponding author. et al., 2011; Socher et al., 2013; Turian et al., 2010; Mikolov et al., 2013b; Bengio et al., 2003). Collobert et al. (2011) developed the SENNA sys- tem that approaches or surpasses the state-of-the- art systems on a variety of sequence labeling tasks for English. Zheng et al. (2013) applied the archi- tecture of Collobert et al. (2011) to Chinese word segmentation and POS tagging, also he proposed a perceptron style algorithm to speed up the train- ing process with negligible loss in performance. Pei et al. (2014) models tag-tag interactions, tag- character interactions and character-character in- teractions based on Zheng et al. (2013). Chen et al. (2015) proposed a gated recursive neural network (GRNN) to explicitly model the combinations of the characters for Chinese word segmentation task. Each neuron in GRNN can be regarded as a differ- ent combination of the input characters. Thus, the whole GRNN has an ability to simulate the design of the sophisticated features in traditional methods. Despite of their success, a limitation of them is that their performances are …

Content truncated to 500 words