Neural Joint Model for Transition-based Chinese Syntactic Analysis

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1204–1214 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1111 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pages 1204–1214 Vancouver, Canada, July 30 - August 4, 2017. c⃝2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-1111 Neural Joint Model for Transition-based Chinese Syntactic Analysis Shuhei Kurita Daisuke Kawahara Graduate School of Informatics, Kyoto University {kurita, dk, kuro}@nlp.ist.i.kyoto-u.ac.jp Sadao Kurohashi Abstract We present neural network-based joint models for Chinese word segmentation, POS tagging and dependency parsing. Our models are the first neural approaches for fully joint Chinese analysis that is known to prevent the error propagation problem of pipeline models. Although word em- beddings play a key role in dependency parsing, they cannot be applied directly to the joint task in the previous work. To address this problem, we propose embed- dings of character strings, in addition to words. Experiments show that our mod- els outperform existing systems in Chinese word segmentation and POS tagging, and perform preferable accuracies in depen- dency parsing. We also explore bi-LSTM models with fewer features. 1 Introduction Dependency parsers have been enhanced by the use of neural networks and embedding vectors (Chen and Manning, 2014; Weiss et al., 2015; Zhou et al., 2015; Alberti et al., 2015; Andor et al., 2016; Dyer et al., 2015). When these dependency parsers process sentences in English and other lan- guages that use symbols for word separations, they can be very accurate. However, for languages that do not contain word separation symbols, de- pendency parsers are used in pipeline processes with word segmentation and POS tagging mod- els, and encounter serious problems because of error propagations. In particular, Chinese word segmentation is notoriously difficult because sen- tences are written without word dividers and Chi- nese words are not clearly defined. Hence, the pipeline of word segmentation, POS tagging and dependency parsing always suffers from word seg- mentation errors. Once words have been wrongly- segmented, word embeddings and traditional one- hot word features, used in dependency parsers, will mistake the precise meanings of the original sentences. As a result, pipeline models achieve dependency scores of around 80% for Chinese. A traditional solution to this error propagation problem is to use joint models. Many Chinese words play multiple grammatical roles with only one grammatical form. Therefore, determining the word boundaries and the subsequent tagging and dependency parsing are closely correlated. Transition-based joint models for Chinese word segmentation, POS tagging and dependency pars- ing are proposed by Hatori et al. (2012) and Zhang et al. (2014). Hatori et al. (2012) state that de- pendency information improves the performances of word segmentation and POS tagging, and de- velop the first transition-based joint word seg- mentation, POS tagging and dependency parsing model. Zhang et al. (2014) expand this and find that both the inter-word dependencies and intra- word dependencies are helpful in word segmenta- tion and POS tagging. Although the models of Hatori et …

Content truncated to 500 words