Joint Chinese Word Segmentation and Part-of-speech Tagging via Multi-channel Attention of Character N-grams

Proceedings of the 28th International Conference on Computational Linguistics, pages 2073–2084 Barcelona, Spain (Online), December 8-13, 2020 2073 Joint Chinese Word Segmentation and Part-of-speech Tagging via Multi-channel Attention of Character N-grams Yuanhe Tian♥, Yan Song♠♥†, Fei Xia♥ ♥University of Washington ♠The Chinese University of Hong Kong (Shenzhen) ♥Shenzhen Research Institute of Big Data ♥{yhtian, fxia}@uw.edu ♠songyan@cuhk.edu.cn Abstract Chinese word segmentation (CWS) and part-of-speech (POS) tagging are two fundamental tasks for Chinese language processing. Previous studies have demonstrated that jointly performing them can be an effective one-step solution to both tasks and this joint task can benefit from a good modeling of contextual features such as n-grams. However, their work on modeling such contextual features is limited to concatenating the features or their embeddings directly with the input embeddings without distinguishing whether the contextual features are important for the joint task in the specific context. Therefore, their models for the joint task could be misled by unimportant contextual information. In this paper, we propose a character-based neural model for the joint task enhanced by multi-channel attention of n-grams. In the attention module, n-gram features are categorized into different groups according to several criteria, and n-grams in each group are weighted and distinguished according to their importance for the joint task in the specific context. To categorize n-grams, we try two criteria in this study, i.e., n-gram frequency and length, so that n-grams having different capabilities of carrying contextual information are discriminatively learned by our proposed attention module. Experimental results on five benchmark datasets for CWS and POS tagging demonstrate that our approach outperforms strong baseline models and achieves state-of-the-art performance on all five datasets.1 1 Introduction Chinese word segmentation (CWS) and part-of-speech (POS) tagging are two fundamental tasks in Chinese natural language processing (NLP). Although they can be treated as two separate tasks in a sequential order, it has been demonstrated by previous studies that processing them jointly in a unified sequence labeling framework could be more effective, where CWS and POS tags are predicted in a single step (Ng and Low, 2004; Jiang et al., 2008; Wang et al., 2011; Sun, 2011; Zeng et al., 2013; Zheng et al., 2013; Zhang et al., 2014; Kurita et al., 2017; Shao et al., 2017; Zhang et al., 2018). In doing so, existing studies mainly focused on incorporating contextual information (e.g., n-grams) as features into their joint taggers, which had been widely used as an effective way to improve model performance especially before neural models were widely used. Although neural models are powerful in modeling long text sequences, external features from larger granular texts are still demonstrated to be useful in existing neural models (Zheng et al., 2013; Kurita et al., 2017; Shao et al., 2017; Zhang et al., 2018). In these models, contextual features are leveraged by directly concatenating their embeddings with the character embeddings in the embedding layer, where all contextual features are treated equally without distinguishing their importance to the joint tagging process in the specific context. However, this concatenation …

Content truncated to 500 words