Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF

Proceedings of the The 8th International Joint Conference on Natural Language Processing, pages 173–183, Taipei, Taiwan, November 27 – December 1, 2017 c⃝2017 AFNLP Character-based Joint Segmentation and POS Tagging for Chinese using Bidirectional RNN-CRF Yan Shao and Christian Hardmeier and J¨org Tiedemann∗ and Joakim Nivre Department of Linguistics and Philology, Uppsala University ∗Department of Modern Languages, University of Helsinki {yan.shao, christian.hardmeier, joakim.nivre}@lingfil.uu.se jorg.tiedemann@helsinki.fi Abstract We present a character-based model for joint segmentation and POS tagging for Chinese. The bidirectional RNN-CRF ar- chitecture for general sequence tagging is adapted and applied with novel vec- tor representations of Chinese characters that capture rich contextual information and sub-character level features. The proposed model is extensively evaluated and compared with a state-of-the-art tag- ger respectively on CTB5, CTB9 and UD Chinese. The experimental results indi- cate that our model is accurate and robust across datasets in different sizes, genres and annotation schemes. We obtain state- of-the-art performance on CTB5, achiev- ing 94.38 F1-score for joint segmentation and POS tagging. 1 Introduction Word segmentation and part-of-speech (POS) tag- ging are core steps for higher-level natural lan- guage processing (NLP) tasks. Given the raw text, segmentation is applied at the very first step and POS tagging is performed on top afterwards. As by convention the words in Chinese are not de- limited by spaces, segmentation is non-trivial, but its accuracy has a significant impact on POS tag- ging. Moreover, POS tags provide useful informa- tion for word segmentation. Thus, modelling word segmentation and POS tagging jointly can out- perform the pipeline models (Ng and Low, 2004; Zhang and Clark, 2008). POS tagging is a typical sequence tagging prob- lem over segmented words, while segmentation also can be modelled as a character-level tagging problem via predicting the labels that identify the word boundaries. Ng and Low (2004) propose a joint model which predicts the combinatory la- bels of segmentation boundaries and POS tags at the character level. Joint segmentation and POS tagging becomes a standard character-based se- quence tagging problem and therefore the general machine learning algorithms for structured predic- tion can be applied. The bidirectional recurrent neural network (RNN) using conditional random fields (CRF) (Lafferty et al., 2001) as the output interface for sentence-level optimisation (BiRNN-CRF) achieves state-of-the-art accuracies on various sequence tagging tasks (Huang et al., 2015; Ma and Hovy, 2016) and outperforms the tradi- tional linear statistical models. RNNs with gated recurrent cells, such as long-short term memory (LSTM) (Hochreiter and Schmidhuber, 1997) and gated recurrent units (GRU) (Cho et al., 2014) are capable of capturing long dependencies and retrieving rich global information. The sequential CRF on top of the recurrent layers ensures that the optimal sequence of tags over the entire sentence is obtained. In this paper, we model joint segmentation and POS tagging as a fully character-based se- quence tagging problem via predicting the com- binatory labels. The BiRNN-CRF architecture is adapted and applied. The Chinese characters are fed into the neural networks as vector representa- tions. In addition to utilising …

Content truncated to 500 words