Stacking Heterogeneous Joint Models of Chinese POS Tagging and Dependency Parsing

Proceedings of COLING 2012: Technical Papers, pages 3071–3088, COLING 2012, Mumbai, December 2012. Stacking Heterogeneous Joint Models of Chinese POS Tagging and Dependency Parsing Meishan Zhang Wanxiang Che Ting Liu∗ Zhenghua Li School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China {mszhang, car, tliu, lzh}@ir.hit.edu.cn ABSTRACT Previous joint models of Chinese part-of-speech (POS) tagging and dependency parsing are extended from either graph- or transition-based dependency models. Our analysis shows that the two models have different error distributions. In addition, integration of graph- and transition-based dependency parsers by stacked learning (stacking) has achieved significant improvements. These motivate us to study the problem of stacking graph- and transition-based joint models. We conduct experiments on Chinese Penn Treebank 5.1 (CTB5.1). The results demonstrate that the guided transition-based joint model obtains better performance than the guided graph-based joint model. Further, we introduce a constituent-based joint model which derives the POS tag sequence and dependency tree from the output of PCFG parsers, and then integrate it into the guided transition-based joint model. Finally, we achieve the best performance on CTB5.1, 94.95% in tagging accuracy and 83.98% in parsing accuracy respectively. TITLE AND ABSTRACT IN CHINESE 采 采 采用 用 用堆 堆 堆方 方 方法 法 法融 融 融合 合 合异 异 异种 种 种的 的 的中 中 中文 文 文词 词 词性 性 性和 和 和依 依 依存 存 存句 句 句法 法 法联 联 联合 合 合模 模 模型 型 型 过去的中文词性和依存句法联合模型基本上都根据基于图的依存句法分析模型或者 基于转移的依存句法分析模型进行拓展而形成的。我们的分析结果表明这两种不同的模型 错误分布并不一样，而且在依存句法中，将基于图的模型和基于转移的模型使用堆方法融 合之后，能够显著的提升依存句法的性能，这些促使我们进一步研究采用堆方法去融合基 于图的和基于转移的词性依存句法联合模型。我们在中文宾州树库5.1版本（CTB5.1）上 进行试验，实验结果表明，相比使用基于图的联合模型为被指导模型，采用转移的联合模 型为被指导模型能取得较好的性能。更进一步，我们介绍了基于短语句法结构的联合模 型，它从一个句子的概率短语文法分析器输出结果中提取句子的词性序列以及依存树结 果，然后我们采用基于短语句法结构的联合模型更进一步指导基于转移的联合模型，最终 我们在CTB5.1的数据上取得了最好结果，词性标注准确率达到94.95%，同时，依存句法 准确率达到83.98%。 KEYWORDS: Chinese POS Tagging, Dependency Parsing, Joint Model, Stacked Learning. KEYWORDS IN CHINESE: 中文词性标注,依存分析,联合模型,堆方法融合学习. ∗Corresponding author 3071 1 Introduction Part-of-speech (POS) tagging and dependency parsing are two fundamental natural language processing (NLP) tasks. Typically, POS tagging is a preprocessing step for dependency parsing, especially in a pipeline architecture. There are two main problems in a pipeline system: (1) Dependency parsing suffers the problem of error propagation; (2) POS tagging cannot exploit useful, important syntactic information for disambiguation. For Chinese POS tagging and dependency parsing, a pipeline system seriously suffers these two problems. The study presented in (Li et al., 2011b) demonstrates the error propagation factor. The authors develop a graph-based joint model for Chinese POS tagging and dependency parsing. The most interesting thing they found is that even with lower tagging accuracy, a joint model could achieve higher parsing accuracy. The work presented in (Hatori et al., 2011) also demonstrates a joint model can largely improve the performance of dependency parsing further. They propose a transition-based joint model for Chinese POS tagging and dependency parsing. Recently, ensemble models have been gained a lot of interests in NLP community. Stacked learning (stacking) (Wolpert, 1992; Breiman, 1996), which is a typical method for ensemble models, has been applied to a number of NLP tasks for its elegance and conciseness, such as Chinese Word segmentation (Sun, 2011), POS tagging (Li et al., 2011a), named entity recognition (Dekai Wu and Carpuat, 2003) and dependency parsing (McDonald, 2006; Nivre and McDonald, 2008; Martins et …

Content truncated to 500 words