Part-of-Speech Tagging for Chinese-English Mixed Texts with Dynamic Features

Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1379–1388, Jeju Island, Korea, 12–14 July 2012. c⃝2012 Association for Computational Linguistics Part-of-Speech Tagging for Chinese-English Mixed Texts with Dynamic Features Jiayi Zhao† Xipeng Qiu‡ Shu Zhang§ Feng Ji‡ Xuanjing Huang‡ School of Computer Science, Fudan University, Shanghai, China † ‡ Fujitsu Research and Development Center, Beijing, China§ zjy.fudan@gmail.com† {xpqiu,fengji,xjhuang}@fudan.edu.cn‡ zhangshu@cn.fujitsu.com § Abstract In modern Chinese articles or conversations, it is very popular to involve a few English words, especially in emails and Internet liter- ature. Therefore, it becomes an important and challenging topic to analyze Chinese-English mixed texts. The underlying problem is how to tag part-of-speech (POS) for the English words involved. Due to the lack of specially annotated corpus, most of the English words are tagged as the oversimplified type, “foreign words”. In this paper, we present a method using dynamic features to tag POS of mixed texts. Experiments show that our method achieves higher performance than traditional sequence labeling methods. Meanwhile, our method also boosts the performance of POS tagging for pure Chinese texts. 1 Introduction Nowadays, Chinese-English mixed texts are prevalent in modern articles or emails. More and more English words are used in Chinese texts as names of organizations, products, terms and abbre- viations, such as “eBay”, “iPhone”, “GDP”, “An- droid” etc. On the other hand, it is also a common phenomenon to use Chinese-English mixed texts in daily conversation, especially in communication among employers in large international corporations. There are some challenges for analyzing Chinese- English mixed texts: 1. How to define the POS tags for English words in these mixed texts. Since the standard of POS tags for English and Chinese are different, we cannot use English POS to tag the English words in mixed texts. 2. Due to lack of annotated corpus for mixed texts, most of the English words are tagged as “for- eign words”, which is oversimplified. So we cannot use them in further processing for the syntactic and semantic analysis. 3. Most English words used in mixed texts are of- ten out-of-vocabulary (OOV), which thus in- creases the difficulties to tag them. Currently, the mainstream method of Chinese POS tagging is joint segmentation & tagging with cross-labels, which can avoid the problem of error propagation and achieve higher performance on both subtasks(Ng and Low, 2004). Each label is the cross- product of a segmentation label and a tagging la- bel, e.g. {B-NN, I-NN, E-NN, S-NN, ...}. The fea- tures are generated by position-based templates on character-level. Since the main part of mixed texts is in Chinese and the role of English word is more like Chinese, we use Chinese POS tags (Xia, 2000) to tag English words. Since the categories of the most commonly used English words are nouns, verbs and adjectives, we can use “NN”, “NR”, “VV”, “VA”, “JJ” to label their POS tags. For the English proper nouns and verbs, there are no significant differences in Chinese and …

Content truncated to 500 words