Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-way Attentions of Auto-analyzed Knowledge

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8286–8296 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 8286 Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-way Attentions of Auto-analyzed Knowledge Yuanhe Tian♥∗, Yan Song♠† , Xiang Ao♣, Fei Xia♥, Xiaojun Quan△, Tong Zhang♦, Yonggang Wang♠ ♥University of Washington, ♠Sinovation Ventures ♣Chinese Academy of Sciences, △Sun Yat-sen University ♦The Hong Kong University of Science and Technology ♥{yhtian, fxia}@uw.edu ♠clksong@gmail.com ♣aoxiang@ict.ac.cn △quanxj3@mail.sysu.edu.cn ♦tongzhang@ust.hk ♠wangyonggang@chuangxin.com Abstract Chinese word segmentation (CWS) and part- of-speech (POS) tagging are important funda- mental tasks for Chinese language processing, where joint learning of them is an effective one-step solution for both tasks. Previous stud- ies for joint CWS and POS tagging mainly follow the character-based tagging paradigm with introducing contextual information such as n-gram features or sentential representa- tions from recurrent neural models. How- ever, for many cases, the joint tagging needs not only modeling from context features but also knowledge attached to them (e.g., syn- tactic relations among words); limited efforts have been made by existing research to meet such needs. In this paper, we propose a neu- ral model named TWASP for joint CWS and POS tagging following the character-based se- quence labeling paradigm, where a two-way at- tention mechanism is used to incorporate both context feature and their corresponding syntac- tic knowledge for each input character. Par- ticularly, we use existing language processing toolkits to obtain the auto-analyzed syntactic knowledge for the context, and the proposed attention module can learn and benefit from them although their quality may not be perfect. Our experiments illustrate the effectiveness of the two-way attentions for joint CWS and POS tagging, where state-of-the-art performance is achieved on five benchmark datasets.1 1 Introduction Chinese word segmentation (CWS) and part-of- speech (POS) tagging are two fundamental and crucial tasks in natural language processing (NLP) for Chinese. The former one aims to find word ∗Partially done as an intern at Sinovation Ventures. †Corresponding author. 1TWASP (code and the best performing models) is re- leased at https://github.com/SVAIGBA/TwASP. Figure 1: An example sentence with CWS and POS tag- ging results, where the ambiguous part (in green color) has dependencies from distant words (in yellow color). boundaries in a sentence and the latter, on the top of segmentation results, assigns a POS tag to each word to indicate its syntactical property in the sen- tence. To effectively perform CWS and POS tag- ging, combining them into a joint task is proved to have better performance than separately conducting the two tasks in a sequence (Ng and Low, 2004). Therefore, many studies were proposed in the past decade for joint CWS and POS tagging (Jiang et al., 2008, 2009; Sun, 2011; Zeng et al., 2013; Zheng et al., 2013; Kurita et al., 2017; Shao et al., 2017; Zhang et al., 2018). These studies, regardless of whether they used conventional approaches (Jiang et al., 2008, 2009; Sun, 2011; Zeng et al., 2013) or deep learning based approaches (Zheng et al., …

Content truncated to 500 words