A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing

A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing Hang Yan, Xipeng Qiu∗, Xuanjing Huang School of Computer Science, Fudan University, China Shanghai Key Laboratory of Intelligent Information Processing, Fudan University, China {hyan19, xpqiu, xjhuang}@fudan.edu.cn Abstract Chinese word segmentation and dependency parsing are two fundamental tasks for Chinese natural language processing. The dependency parsing is defined at the word-level. Therefore word segmentation is the precondition of dependencyparsing, which makes dependency parsing suffer from error propagation and unable to directly make use of character-level pre-trained language models (such as BERT). In this paper, we propose a graph-based model to integrate Chinese word segmentation and dependency parsing. Different from previous transition-based joint models, our proposed model is more concise, which results in fewer efforts of feature engineering. Our graph-based joint model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing. Additionally, when BERT is combined, our model can substan- tially reduce the performance gap of depen- dency parsing between joint models and gold-segmented word-based models. Our code is publicly available at https://github. com/fastnlp/JointCwsParser. 1 Introduction Unlike English, Chinese sentences consist of continuous characters and lack obvious bound- aries between Chinese words. Words are usually regarded as the minimum semantic unit, there- fore Chinese word segmentation (CWS) becomes a preliminary pre-process step for downstream Chinese natural language processing (NLP). For example, the fundamental NLP task, dependency ∗Corresponding author. parsing, is usually defined at the word-level. To parse a Chinese sentence, the process is usually performed in the following pipeline method: word segmentation, part-of-speech (POS) tagging, and dependency parsing. However, the pipeline method always suffers from the following limitations: (1) Error Propagation. In the pipeline method, once some words are wrongly segmented, the subsequent POS tagging and parsing will also make mistakes. As a result, pipeline models achieve dependency scores of around 75 ∼ 80% (Kurita et al., 2017). (2) Knowledge Sharing. These three tasks (word segmentation, POS tagging, and dependency parsing) are strongly related. The criterion of CWS also depends on the word’s gram- matical role in a sentence. Therefore, the knowledge learned from these three tasks can be shared. The knowledge of one task can help others. However, the pipeline method separately trains three models, each for a sin- gle task, and cannot fully exploit the shared knowledge among the three tasks. A traditional solution to this error propagation problem is to use joint models (Hatori et al., 2012; Zhang et al., 2014; Kurita et al., 2017). These previous joint models mainly adopted a transition- based parsing framework to integrate the word seg- mentation, POS tagging, and dependency parsing. Based on standard sequential shift-reduce transi- tions, they design some extra actions for word segmentation and POS tagging. Although these joint models achieved better performance than the pipeline model, they still suffer from two limitations: (1) The first is the huge search space. Com- pared with word-level transition parsing, 78 Transactions of the Association for Computational Linguistics, vol. 8, pp. 78–92, 2020. …

Content truncated to 500 words