A Separately Passive-Aggressive Training Algorithm for Joint POS Tagging and Dependency Parsing

Proceedings of COLING 2012: Technical Papers, pages 1681–1698, COLING 2012, Mumbai, December 2012. A Separately Passive-Aggressive Training Algorithm for Joint POS Tagging and Dependency Parsing Zhenghua Li1, Min Zhang2, Wanxiang Che1, Ting Liu1∗ (1) Research Center for Social Computing and Information Retrieval, Harbin Institute of Technology, China (2) Institute for Infocomm Research, Singapore {lzh,car,tliu}@ir.hit.edu.cn, mzhang@i2r.a-star.edu.sg ABSTRACT Recent study shows that parsing accuracy can be largely improved by the joint optimization of part-of-speech (POS) tagging and dependency parsing. However, the POS tagging task does not benefit much from the joint framework. We argue that the fundamental reason behind is because the POS features are overwhelmed by the syntactic features during the joint optimization, and the joint models only prefer such POS tags that are favourable solely from the parsing viewpoint. To solve this issue, we propose a separately passive-aggressive learning algorithm (SPA), which is designed to separately update the POS features weights and the syntactic feature weights under the joint optimization framework. The proposed SPA is able to take advantage of previous joint optimization strategies to significantly improve the parsing accuracy, but also overcome their shortages to significantly boost the tagging accuracy by effectively solving the syntax-insensitive POS ambiguity issues. Experiments on the Chinese Penn Treebank 5.1 (CTB5) and the English Penn Treebank (PTB) demonstrate the effectiveness of our proposed methodology and empirically verify our observations as discussed above. We achieve the best tagging and parsing accuracies on both datasets, 94.60% in tagging accuracy and 81.67% in parsing accuracy on CTB5, and 97.62% and 93.52% on PTB. KEYWORDS: Part-of-speech Tagging, Dependency Parsing, Joint Models, Separately Passive- aggressive Algorithm. ∗ Corresponding author 1681 1 Introduction Given an input sentence of n words, denoted by x = w1...wn, part-of-speech (POS) tagging aims to find an optimal tag sequence t = t1...tn, where ti ∈ � (1 ≤ i ≤ n) and � is a predefined tag set. POS tags are designed to represent word classes so that words of the same POS tag play a similar role in syntactic structures. The size of � is usually much less than the vocabulary size. Typically, POS tagging is treated as a sequence labeling problem, and has been previously addressed by machine learning algorithms, such as maximum-entropy (Ratnaparkhi, 1996), conditional random fields (CRF) (Lafferty et al., 2001) and perceptron (Collins, 2002). Figure 1 gives an example sentence from Penn Chinese Treebank 5.1 (CTB5). The lowest three rows present the n-best POS tags for each word, produced by a state-of- the-art CRF model. Looking at the 1-best POS tags, we can see that the CRF model makes four errors, i.e. de/DEC→DEG, ouwen/NR→NN, xiaoli/VV→NN, and liwupudui/NR→NN. In fact, (DEC,DEG) and (NN,VV) ambiguities, which usually require long-distance syntactic knowledge to resolve, are very difficult for the sequential labeling models. �1 �2 193 �4 �5 '�6 �(7 H�8 )if�9 �10 gang man 19 sui de ouwen xianzai xiaoli liwupudui . just turned 19 years old who Owen now plays for Liverpool . AD VV CD M DEG NN NT NN …

Content truncated to 500 words