A Joint Model for Unsupervised Chinese Word Segmentation

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 854–863, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics A Joint Model for Unsupervised Chinese Word Segmentation Miaohong Chen Baobao Chang Wenzhe Pei Key Laboratory of Computational Linguistics, Ministry of Education School of Electronics Engineering and Computer Science, Peking University Beijing, P.R.China, 100871 miaohong-chen@foxmail.com,{chbb,peiwenzhe}@pku.edu.cn Abstract In this paper, we propose a joint model for unsupervised Chinese word segmentation (CWS). Inspired by the “products of ex- perts” idea, our joint model firstly com- bines two generative models, which are word-based hierarchical Dirichlet process model and character-based hidden Markov model, by simply multiplying their proba- bilities together. Gibbs sampling is used for model inference. In order to further combine the strength of goodness-based model, we then integrated nVBE into our joint model by using it to initializing the Gibbs sampler. We conduct our experi- ments on PKU and MSRA datasets pro- vided by the second SIGHAN bakeoff. Test results on these two datasets show that the joint model achieves much bet- ter results than all of its component mod- els. Statistical significance tests also show that it is significantly better than state- of-the-art systems, achieving the highest F-scores. Finally, analysis indicates that compared with nVBE and HDP, the joint model has a stronger ability to solve both combinational and overlapping ambigui- ties in Chinese word segmentation. 1 Introduction Unlike English and many other western languages, there are no explicit word boundaries in Chinese sentences. Therefore, word segmentation is a cru- cial first step for many Chinese language process- ing tasks such as syntactic parsing, information re- trieval and machine translation. A great deal of su- pervised methods have been proposed for Chinese word segmentation. While successful, they re- quire manually labeled resources and often suffer from issues like poor domain adaptability. Thus, unsupervised word segmentation methods are still attractive to researchers due to its independence on domain and manually labeled corpora. Previous unsupervised approaches to word seg- mentation can be roughly classified into two types. The first type uses carefully designed goodness measure to identify word candidates. Popular goodness measures include description length gain (DLG) (Kit and Wilks, 1999), accessor variety (AV) (Feng et al., 2004), boundary entropy (BE) (Jin and Tanaka-Ishii, 2006) and normalized vari- ation of branching entropy (nVBE) (Magistry and Sagot, 2012) etc. Goodness measure based model is not segmentation model in a very strict mean- ing and is actually strong in generating word list without supervision. It inherently lacks capabil- ity to deal with ambiguous string, which is one of main sources of segmentation errors and has been extensively explored in supervised Chinese word segmentation. The second type focuses on designing sophis- ticated statistical model, usually nonparametric Bayesian models, to find the segmentation with highest posterior probability, given the observed character sequences. Typical statistical mod- els includes Hierarchical Dirichlet process (HDP) model (Goldwater et al., 2009), Nested Pitman- Yor process (NPY) model (Mochihashi et al., 2009) etc, which are actually nonparametric lan- …

Content truncated to 500 words