Gated Recursive Neural Network for Chinese Word Segmentation

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1744–1753, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Gated Recursive Neural Network for Chinese Word Segmentation Xinchi Chen, Xipeng Qiu∗, Chenxi Zhu, Xuanjing Huang Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, China {xinchichen13,xpqiu,czhu13,xjhuang}@fudan.edu.cn Abstract Recently, neural network models for natu- ral language processing tasks have been in- creasingly focused on for their ability of al- leviating the burden of manual feature en- gineering. However, the previous neural models cannot extract the complicated fea- ture compositions as the traditional meth- ods with discrete features. In this paper, we propose a gated recursive neural net- work (GRNN) for Chinese word segmen- tation, which contains reset and update gates to incorporate the complicated com- binations of the context characters. Since GRNN is relative deep, we also use a supervised layer-wise training method to avoid the problem of gradient diffusion. Experiments on the benchmark datasets show that our model outperforms the pre- vious neural network models as well as the state-of-the-art methods. 1 Introduction Unlike English and other western languages, Chi- nese do not delimit words by white-space. There- fore, word segmentation is a preliminary and im- portant pre-process for Chinese language process- ing. Most previous systems address this problem by treating this task as a sequence labeling prob- lem and have achieved great success. Due to the nature of supervised learning, the performance of these models is greatly affected by the design of features. These features are explicitly represented by the different combinations of context charac- ters, which are based on linguistic intuition and sta- tistical information. However, the number of fea- tures could be so large that the result models are too large to use in practice and prone to overfit on training corpus. ∗Corresponding author. Rainy 下 雨 Day 天 Ground 地 面 Accumulated water 积 水 M E S B Figure 1: Illustration of our model for Chinese word segmentation. The solid nodes indicate the active neurons, while the hollow ones indicate the suppressed neurons. Specifically, the links denote the information flow, where the solid edges de- note the acceptation of the combinations while the dashed edges means rejection of that. As shown in the right figure, we receive a score vector for tag- ging target character “地” by incorporating all the combination information. Recently, neural network models have been in- creasingly focused on for their ability to minimize the effort in feature engineering. Collobert et al. (2011) developed a general neural network archi- tecture for sequence labeling tasks. Following this work, many methods (Zheng et al., 2013; Pei et al., 2014; Qi et al., 2014) applied the neural net- work to Chinese word segmentation and achieved a performance that approaches the state-of-the-art methods. However, these neural models just concatenate the embeddings of the context characters, and feed them into neural network. …

Content truncated to 500 words