Max-Margin Tensor Neural Network for Chinese Word Segmentation

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 293–303, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Max-Margin Tensor Neural Network for Chinese Word Segmentation Wenzhe Pei Tao Ge Baobao Chang∗ Key Laboratory of Computational Linguistics, Ministry of Education School of Electronics Engineering and Computer Science, Peking University Beijing, P.R.China, 100871 {peiwenzhe,getao,chbb}@pku.edu.cn Abstract Recently, neural network models for nat- ural language processing tasks have been increasingly focused on for their ability to alleviate the burden of manual feature engineering. In this paper, we propose a novel neural network model for Chinese word segmentation called Max-Margin Tensor Neural Network (MMTNN). By exploiting tag embeddings and tensor- based transformation, MMTNN has the ability to model complicated interactions between tags and context characters. Fur- thermore, a new tensor factorization ap- proach is proposed to speed up the model and avoid overfitting. Experiments on the benchmark dataset show that our model achieves better performances than previ- ous neural network models and that our model can achieve a competitive perfor- mance with minimal feature engineering. Despite Chinese word segmentation being a specific case, MMTNN can be easily generalized and applied to other sequence labeling tasks. 1 Introduction Unlike English and other western languages, Chi- nese do not delimit words by white-space. There- fore, word segmentation is a preliminary and im- portant pre-process for Chinese language process- ing. Most previous systems address this problem by treating this task as a sequence labeling prob- lem where each character is assigned a tag indi- cating its position in the word. These systems are effective because researchers can incorporate a large body of handcrafted features into the models. However, the ability of these models is restricted ∗Corresponding author by the design of features and the number of fea- tures could be so large that the result models are too large for practical use and prone to overfit on training corpus. Recently, neural network models have been in- creasingly focused on for their ability to mini- mize the effort in feature engineering. Collobert et al. (2011) developed the SENNA system that ap- proaches or surpasses the state-of-the-art systems on a variety of sequence labeling tasks for English. Zheng et al. (2013) applied the architecture of Collobert et al. (2011) to Chinese word segmenta- tion and POS tagging and proposed a perceptron- style algorithm to speed up the training process with negligible loss in performance. Workable as previous neural network models seem, a limitation of them to be pointed out is that the tag-tag interaction, tag-character inter- action and character-character interaction are not well modeled. In conventional feature-based lin- ear (log-linear) models, these interactions are ex- plicitly modeled as features. Take phrase “打篮 球(play basketball)” as an example, assuming we are labeling character C0=“篮”, possible features could be: f1 = � 1 C−1=“打” and C1=“球” and y0=“B” 0 else f2 = � 1 C0=“篮” and y0=“B” and y−1=“S” 0 else To capture more interactions, researchers have de- signed a large number of features …

Content truncated to 500 words