Joint Training and Decoding Using Virtual Nodes for Cascaded Segmentation and Tagging Tasks

Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 187–195, MIT, Massachusetts, USA, 9-11 October 2010. c⃝2010 Association for Computational Linguistics Joint Training and Decoding Using Virtual Nodes for Cascaded Segmentation and Tagging Tasks Xian Qian, Qi Zhang, Yaqian Zhou, Xuanjing Huang, Lide Wu School of Computer Science, Fudan University 825 Zhangheng Road, Shanghai, P.R.China {qianxian, qz, zhouyaqian, xjhuang, ldwu}@fudan.edu.cn Abstract Many sequence labeling tasks in NLP require solving a cascade of segmentation and tag- ging subtasks, such as Chinese POS tagging, named entity recognition, and so on. Tradi- tional pipeline approaches usually suffer from error propagation. Joint training/decoding in the cross-product state space could cause too many parameters and high inference complex- ity. In this paper, we present a novel method which integrates graph structures of two sub- tasks into one using virtual nodes, and per- forms joint training and decoding in the fac- torized state space. Experimental evaluations on CoNLL 2000 shallow parsing data set and Fourth SIGHAN Bakeoff CTB POS tagging data set demonstrate the superiority of our method over cross-product, pipeline and can- didate reranking approaches. 1 Introduction There is a typical class of sequence labeling tasks in many natural language processing (NLP) applica- tions, which require solving a cascade of segmenta- tion and tagging subtasks. For example, many Asian languages such as Japanese and Chinese which do not contain explicitly marked word boundaries, word segmentation is the preliminary step for solv- ing part-of-speech (POS) tagging problem. Sen- tences are firstly segmented into words, then each word is assigned with a part-of-speech tag. Both syntactic parsing and dependency parsing usually start with a textual input that is tokenized, and POS tagged. The most commonly approach solves cascaded subtasks in a pipeline, which is very simple to im- plement and allows for a modular approach. While, the key disadvantage of such method is that er- rors propagate between stages, significantly affect- ing the quality of the final results. To cope with this problem, Shi and Wang (2007) proposed a rerank- ing framework in which N-best segment candidates generated in the first stage are passed to the tag- ging model, and the final output is the one with the highest overall segmentation and tagging probabil- ity score. The main drawback of this method is that the interaction between tagging and segmentation is restricted by the number of candidate segmentation outputs. Razvan C. Bunescu (2008) presented an improved pipeline model in which upstream subtask outputs are regarded as hidden variables, together with their probabilities are used as probabilistic fea- tures in the downstream subtasks. One shortcom- ing of this method is that calculation of marginal probabilities of features may be inefficient and some approximations are required for fast computation. Another disadvantage of these two methods is that they employ separate training and the segmentation model could not take advantages of tagging infor- mation in the training procedure. On the other hand, joint learning and decoding using cross-product of segmentation states and tag- ging states does …

Content truncated to 500 words