Transition-Based Neural Word Segmentation

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 421–431, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Transition-Based Neural Word Segmentation Meishan Zhang1 and Yue Zhang2 and Guohong Fu1 1. School of Computer Science and Technology, Heilongjiang University, Harbin, China 2. Singapore University of Technology and Design mason.zms@gmail.com, yue zhang@sutd.edu.sg, ghfu@hotmail.com Abstract Character-based and word-based methods are two main types of statistical models for Chinese word segmentation, the for- mer exploiting sequence labeling models over characters and the latter typically ex- ploiting a transition-based model, with the advantages that word-level features can be easily utilized. Neural models have been exploited for character-based Chi- nese word segmentation, giving high accu- racies by making use of external character embeddings, yet requiring less feature en- gineering. In this paper, we study a neu- ral model for word-based Chinese word segmentation, by replacing the manually- designed discrete features with neural fea- tures in a word-based segmentation frame- work. Experimental results demonstrate that word features lead to comparable per- formances to the best systems in the litera- ture, and a further combination of discrete and neural features gives top accuracies. 1 Introduction Statistical word segmentation methods can be cat- egorized character-based (Xue, 2003; Tseng et al., 2005) and word-based (Andrew, 2006; Zhang and Clark, 2007) approaches. The former casts word segmentation as a sequence labeling problem, us- ing segmentation tags on characters to mark their relative positions inside words. The latter, in con- trast, ranks candidate segmented outputs directly, extracting both character and full-word features. An influential character-based word segmenta- tion model (Peng et al., 2004; Tseng et al., 2005) uses B/I/E/S labels to mark a character as the be- ginning, internal (neither beginning nor end), end and only-character (both beginning and end) of a character-based word-based discrete Peng et al. (2004) Andrew (2006) Tseng et al. (2005) Zhang and Clark (2007) neural Zheng et al. (2013) this work Chen et al. (2015b) Figure 1: Word segmentation methods. word, respectively, employing conditional random field (CRF) to model the correspondence between the input character sequence and output label se- quence. For each character, features are extracted from a five-character context window and a two- label history window. Subsequent work explores different label sets (Zhao et al., 2006), feature sets (Shi and Wang, 2007) and semi-supervised learn- ing (Sun and Xu, 2011), reporting state-of-the-art accuracies. Recently, neural network models have been in- vestigated for the character tagging approach. The main idea is to replace manual discrete features with automatic real-valued features, which are de- rived automatically from distributed character rep- resentations using neural networks. In particular, convolution neural network1 (Zheng et al., 2013), tensor neural network (Pei et al., 2014), recur- sive neural network (Chen et al., 2015a) and long- short-term-memory (LSTM) (Chen et al., 2015b) have been used to derive neural feature represen- tations from input word sequences, which are fed into a CRF inference layer. In this paper, we investigate the effectiveness of word embedding features for neural …

Content truncated to 500 words