Coupled Sequence Labeling on Heterogeneous Annotations: POS Tagging as a Case Study

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1783–1792, Beijing, China, July 26-31, 2015. c⃝2015 Association for Computational Linguistics Coupled Sequence Labeling on Heterogeneous Annotations: POS Tagging as a Case Study Zhenghua Li, Jiayuan Chao, Min Zhang∗, Wenliang Chen (1) Soochow University (2) Collaborative Innovation Center of Novel Software Technology and Industrialization Jiangsu Province, China {zhli13,minzhang,wlchen}@suda.edu.cn; china cjy@163.com Abstract In order to effectively utilize multiple datasets with heterogeneous annotations, this paper proposes a coupled sequence labeling model that can directly learn and infer two heterogeneous annotations simultaneously, and to facilitate discussion we use Chinese part-of- speech (POS) tagging as our case study. The key idea is to bundle two sets of POS tags together (e.g. “[NN, n]”), and build a conditional random field (CRF) based tagging model in the enlarged space of bundled tags with the help of ambiguous labelings. To train our model on two non-overlapping datasets that each has only one-side tags, we transform a one-side tag into a set of bundled tags by considering all possible mappings at the missing side and derive an objective function based on ambiguous labelings. The key advantage of our coupled model is to provide us with the flexibility of 1) incorporating joint features on the bundled tags to implicitly learn the loose mapping between heterogeneous annotations, and 2) exploring separate features on one-side tags to overcome the data sparseness problem of using only bundled tags. Experiments on benchmark datasets show that our coupled model significantly outperforms the state-of- the-art baselines on both one-side POS tagging and annotation conversion tasks. The codes and newly annotated data are released for non-commercial usage.1 ∗Correspondence author. 1http://hlt.suda.edu.cn/˜zhli 1 Introduction The scale of available labeled data significantly affects the performance of statistical data-driven models. As a widely-used structural classification problem, sequence labeling is prone to suffer from the data sparseness issue. However, the heavy cost of manual annotation typically limits one labeled resource in both scale and genre. As a promising research line, semi-supervised learning for sequence labeling has been exten- sively studied. Huang et al. (2009) show that standard self-training can boost the performance of a simple hidden Markov model (HMM) based part-of-speech (POS) tagger. Søgaard (2011) ap- ply tri-training to English POS tagging, boost- ing accuracy from 97.27% to 97.50%. Sun and Uszkoreit (2012) derive word clusters from large- scale unlabeled data as extra features for Chi- nese POS tagging. Recently, the use of natural annotation has becomes a hot topic in Chinese word segmentation (Jiang et al., 2013; Liu et al., 2014; Yang and Vozila, 2014). The idea is to derive segmentation boundaries from implicit information encoded in web texts, such as anchor texts and punctuation marks, and use them as partially labeled training data in sequence labeling models. The existence of multiple annotated resources opens another door for alleviating data sparse- ness. For example, Penn Chinese Treebank (CTB) contains about 20 thousand sentences annotated with word boundaries, …

Content truncated to 500 words