Unsupervised Overlapping Feature Selection for Conditional Random Fields Learning in Chinese Word Segmentation

Unsupervised Overlapping Feature Selection for Conditional Random Fields Learning in Chinese Word Segmentation Ting-hao Yang Institute of Information Science Academia Sinica tinghaoyang@iis.sinica.edu.tw Tian-Jian Jiang Department of Computer Science National Tsing-Hua University tmjiang@iis.sinica.edu.tw Chan-hung Kuo Institute of Information Science Academia Sinica laybow@iis.sinica.edu.tw Richard Tzong-han Tsai Department of Computer Science & Engineering Yuan Ze University thtsai@saturn.yzu.edu.tw Wen-lian Hsu Institute of Information Science Academia Sinica hsu@iis.sinica.edu.tw Abstract This work represents several unsupervised feature selections based on frequent strings that help improve conditional random fields (CRF) model for Chinese word segmentation (CWS). These features include character-based N-gram (CNG), Accessor Variety based string (AVS), and Term Contributed Frequency (TCF) with a specific manner of boundary overlapping. For the experiment, the baseline is the 6-tag, a state-of-the-art labeling scheme of CRF-based CWS; and the data set is acquired from SIGHAN CWS bakeoff 2005. The experiment results show that all of those features improve our system’s F1 measure (F) and Recall of Out-of-Vocabulary (ROOV). In particular, the feature collections which contain AVS feature outperform other types of features in terms of F, whereas the feature collections containing TCB/TCF information has better ROOV. Keywords: Word Segmentation, Unsupervised Feature Selection, Conditional Random Fields 1. Introduction Many intelligent text processing tasks such as information retrieval, text-to-speech and 109 machine translation assume the ready availability of a tokenization into words, which is relatively straightforward in languages with word delimiters (e.g. space), while a little difficult for Asian languages such as Chinese and Japanese. 1.1 Background Chinese word segmentation (CWS) is an essential pre-work for Chinese text processing applications and it has been an active area of research in computational linguistics for two decades. SIGHAN, the Special Interest Group for Chinese Language Processing of the Association for Computational Linguistics, conducted five word segmentation bakeoffs (Sproat and Emerson, 2003; Emerson, 2005; Levow, 2006; Jin and Chen, 2007; Zhao and Liu, 2010). After years of intensive researches, CWS has achieved high precision, but the issue of out-of-vocabulary word handling still remains. 1.2 The State of the Art of CWS Traditional approaches for CWS adopted dictionary and rules to segment unlabeled texts (c.f. Ma and Chen, 2003). In recent years, the mainstream is to use statistical machine learning models, especially the Conditional Random Fields (CRF) (Lafferty et al, 2001), which shows a moderate performance for sequential labeling problem and achieves competitive results with character position based methods (Zhao et al., 2010). 1.3 Unsupervised CRF Feature Selections for CWS For incorporating unsupervised feature selections into character position based CRF for CWS, Zhao and Kit (2006; 2007) tried strings based on Accessor Variety (AV), which was developed by Feng et al. (2004), and co-occurrence strings (COS). Jiang et al. (2010) applied a feature similar to COS, called Term Contributed Boundary (TCB). Tsai (2010) employ statistical association measures non-parametrically through a natural but novel feature representation scheme. Those unsupervised feature selection are based on frequent strings extracted automatically from unlabeled corpora. They are suitable for closed training evaluation that any external resource or extra information is not allowed. …

Content truncated to 500 words