An Error-Driven Word-Character Hybrid Model for Joint Chinese Word Segmentation and POS Tagging

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 513–521, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP An Error-Driven Word-Character Hybrid Model for Joint Chinese Word Segmentation and POS Tagging Canasai Kruengkrai†‡ and Kiyotaka Uchimoto‡ and Jun’ichi Kazama‡ Yiou Wang‡ and Kentaro Torisawa‡ and Hitoshi Isahara†‡ †Graduate School of Engineering, Kobe University 1-1 Rokkodai-cho, Nada-ku, Kobe 657-8501 Japan ‡National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto 619-0289 Japan {canasai,uchimoto,kazama,wangyiou,torisawa,isahara}@nict.go.jp Abstract In this paper, we present a discriminative word-character hybrid model for joint Chi- nese word segmentation and POS tagging. Our word-character hybrid model offers high performance since it can handle both known and unknown words. We describe our strategies that yield good balance for learning the characteristics of known and unknown words and propose an error- driven policy that delivers such balance by acquiring examples of unknown words from particular errors in a training cor- pus. We describe an efficient framework for training our model based on the Mar- gin Infused Relaxed Algorithm (MIRA), evaluate our approach on the Penn Chinese Treebank, and show that it achieves supe- rior performance compared to the state-of- the-art approaches reported in the litera- ture. 1 Introduction In Chinese, word segmentation and part-of-speech (POS) tagging are indispensable steps for higher- level NLP tasks. Word segmentation and POS tag- ging results are required as inputs to other NLP tasks, such as phrase chunking, dependency pars- ing, and machine translation. Word segmenta- tion and POS tagging in a joint process have re- ceived much attention in recent research and have shown improvements over a pipelined fashion (Ng and Low, 2004; Nakagawa and Uchimoto, 2007; Zhang and Clark, 2008; Jiang et al., 2008a; Jiang et al., 2008b). In joint word segmentation and the POS tag- ging process, one serious problem is caused by unknown words, which are defined as words that are not found in a training corpus or in a sys- tem’s word dictionary1. The word boundaries and the POS tags of unknown words, which are very difficult to identify, cause numerous errors. The word-character hybrid model proposed by Naka- gawa and Uchimoto (Nakagawa, 2004; Nakagawa and Uchimoto, 2007) shows promising properties for solving this problem. However, it suffers from structural complexity. Nakagawa (2004) described a training method based on a word-based Markov model and a character-based maximum entropy model that can be completed in a reasonable time. However, this training method is limited by the generatively-trained Markov model in which in- formative features are hard to exploit. In this paper, we overcome such limitations concerning both efficiency and effectiveness. We propose a new framework for training the word- character hybrid model based on the Margin Infused Relaxed Algorithm (MIRA) (Crammer, 2004; Crammer et al., 2005; McDonald, 2006). We describe k-best decoding for our hybrid model and design its loss function and the features appro- priate for our task. In our word-character hybrid model, allowing the model to learn the characteristics of …

Content truncated to 500 words