Joint Chinese Word Segmentation and POS Tagging on Heterogeneous Annotated Corpora with Multiple Task Learning

Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 658–668, Seattle, Washington, USA, 18-21 October 2013. c⃝2013 Association for Computational Linguistics Joint Chinese Word Segmentation and POS Tagging on Heterogeneous Annotated Corpora with Multiple Task Learning Xipeng Qiu, Jiayi Zhao, Xuanjing Huang Fudan University, 825 Zhangheng Road, Shanghai, China xpqiu@fudan.edu.cn, zjy.fudan@gmail.com, xjhuang@fudan.edu.cn Abstract Chinese word segmentation and part-of- speech tagging (S&T) are fundamental steps for more advanced Chinese language processing tasks. Recently, it has at- tracted more and more research interests to exploit heterogeneous annotation cor- pora for Chinese S&T. In this paper, we propose a unified model for Chinese S&T with heterogeneous annotation corpora. We first automatically construct a loose and uncertain mapping between two rep- resentative heterogeneous corpora, Penn Chinese Treebank (CTB) and PKU’s Peo- ple’s Daily (PPD). Then we regard the Chinese S&T with heterogeneous corpora as two “related” tasks and train our model on two heterogeneous corpora simultane- ously. Experiments show that our method can boost the performances of both of the heterogeneous corpora by using the shared information, and achieves significant im- provements over the state-of-the-art meth- ods. 1 Introduction Currently, most of statistical natural language processing (NLP) systems rely heavily on manu- ally annotated resources to train their statistical models. The more of the data scale, the better the performance will be. However, the costs are extremely expensive to build the large scale re- sources for some NLP tasks. Even worse, the ex- isting resources are often incompatible even for a same task and the annotation guidelines are usu- ally different for different projects, since there are many underlying linguistic theories which explain the same language with different per- spectives. As a result, there often exist multi- ple heterogeneous annotated corpora for a same task with vastly different and incompatible an- notation philosophies. These heterogeneous re- sources are waste on some level if we cannot fully exploit them. However, though most of statistical NLP methods are not bound to specific annota- tion standards, almost all of them cannot deal simultaneously with the training data with different and incompatible annotation. The co-existence of heterogeneous annotation data therefore presents a new challenge to utilize these resources. The problem of incompatible annotation stan- dards is very serious for many tasks in NLP, especially for Chinese word segmentation and part-of-speech (POS) tagging (Chinese S&T). In Chinese S&T, the annotation standards are of- ten incompatible for two main reasons. One is that there is no widely accepted segmentation standard due to the lack of a clear definition of Chinese words. Another is that there are no morphology for Chinese word so that there are many ambiguities to tag the parts-of-speech for Chinese word. For example, the two commonly- used corpora, PKU’s People’s Daily (PPD) (Yu et al., 2001) and Penn Chinese Treebank (CTB) (Xia, 2000), use very different segmentation and POS tagging standards. For example, in Table 1, it is very different to annotate the sentence “刘翔进入中国区总 决赛 (Liu Xiang reaches the national final in …

Content truncated to 500 words