Fast Coupled Sequence Labeling on Heterogeneous Annotations via Context-aware Pruning

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 753–762, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Fast Coupled Sequence Labeling on Heterogeneous Annotations via Context-aware Pruning Zhenghua Li, Jiayuan Chao, Min Zhang∗, Jiwen Yang Soochow University, Suzhou, China {zhli13,minzhang,jwyang}@suda.edu.cn, chaojiayuan.china@gmail.com Abstract The recently proposed coupled sequence label- ing is shown to be able to effectively exploit multiple labeled data with heterogeneous an- notations but suffer from severe inefficiency problem due to the large bundled tag space (Li et al., 2015). In their case study of part-of- speech (POS) tagging, Li et al. (2015) man- ually design context-free tag-to-tag mapping rules with a lot of effort to reduce the tag space. This paper proposes a context-aware prun- ing approach that performs token-wise con- straints on the tag space based on contextual evidences, making the coupled approach effi- cient enough to be applied to the more complex task of joint word segmentation (WS) and POS tagging for the first time. Experiments show that using the large-scale People Daily as auxiliary heterogeneous data, the coupled approach can improve F-score by 95.55 − 94.88 = 0.67% on WS, and by 90.58 − 89.49 = 1.09% on joint WS&POS on Penn Chinese Treebank. All codes are released at http://hlt.suda.edu.cn/~zhli. 1 Introduction In statistical natural language processing, manually labeled data is inevitable for model supervision, but is also very expensive to build. However, due to the long-debated differences in underlying linguistic theories or emphasis of application, there often exist multiple labeled corpora for the same or similar tasks following different annotation guidelines (Jiang et ∗Correspondence author Especially our nation economy declines . CTB 特别是/AD 我/PN 国/NN 经济/NN 下滑/VV 。/PU PD 特别/d 是/v 我国/n 经济/n 下滑/v 。/w Table 1: An example of heterogeneous annotations. al., 2009). For instance, in Chinese language pro- cessing, Penn Chinese Treebank version 5 (CTB5) is a widely used benchmark data and contains about 20 thousand sentences annotated with word boundaries, part-of-speech (POS) tags, and syntactic structures (Xue et al., 2005; Xia, 2000), whereas People’s Daily corpus (PD)1 is a large-scale corpus annotated with words and POS tags, containing about 300 thou- sand sentences from the first half of 1998 of People’s Daily newspaper (Yu et al., 2003). Table 1 gives an example with both CTB and PD annotations. We can see that CTB and PD differ in both word boundary standards and POS tag sets. Previous work on exploiting heterogeneous data mainly focuses on indirect guide-feature methods. The basic idea is to use one resource to generate extra guide features on another resource (Jiang et al., 2009; Sun and Wan, 2012), which is similar to stacked learning (Nivre and McDonald, 2008). Li et al. (2015) propose a coupled sequence labeling approach that can directly learn and predict two het- erogeneous annotations simultaneously. The basic idea is to transform a single-side tag into a set of bundled tags for weak supervision based on the idea of ambiguous labeling. Due to the huge size of the …

Content truncated to 500 words