Nonparametric Bayesian Semi-supervised Word Segmentation

Nonparametric Bayesian Semi-supervised Word Segmentation Ryo Fujii Ryo Domoto Hakuhodo Inc. R&D Division 5-3-1 Akasaka, Minato-ku, Tokyo {ryo.b.fujii,ryo.domoto}@hakuhodo.co.jp Daichi Mochihashi The Institute of Statistical Mathematics 10-3 Midori-cho, Tachikawa city, Tokyo daichi@ism.ac.jp Abstract This paper presents a novel hybrid genera- tive/discriminative model of word segmenta- tion based on nonparametric Bayesian meth- ods. Unlike ordinary discriminative word seg- mentation which relies only on labeled data, our semi-supervised model also leverages a huge amounts of unlabeled text to automat- ically learn new “words”, and further con- strains them by using a labeled data to seg- ment non-standard texts such as those found in social networking services. Specifically, our hybrid model combines a discriminative classifier (CRF; Lafferty et al. (2001) and unsupervised word segmentation (NPYLM; Mochihashi et al. (2009)), with a transparent exchange of information between these two model structures within the semi- supervised framework (JESS-CM; Suzuki and Isozaki (2008)). We confirmed that it can appropriately segment non-standard texts like those in Twitter and Weibo and has nearly state-of-the-art accuracy on standard datasets in Japanese, Chinese, and Thai. 1 Introduction For any unsegmented language, especially East Asian languages such as Chinese, Japanese and Thai, word segmentation is almost an inevitable first step in natural language processing. In fact, it is be- coming increasingly important lately because of the growing interest in processing user-generated me- dia, such as Twitter and blogs. Texts in such media are often written in a colloquial style that contains many new words and expressions that are not present in any existing dictionaries. Since such words are theoretically infinite in number, we need to lever- age unsupervised learning to automatically identify them in corpora. For this purpose, ordinary supervised learning is clearly unsatisfactory; even hand-crafted dictionar- ies will not suffice because functional expressions more complex than simple nouns need to be recog- nized through their relationship with other words in text, which also might be unknown in advance. Pre- vious studies of this issue used character and word information in the framework of supervised learning (Kruengkrai et al., 2009; Sun et al., 2009; Sun and Xu, 2011). However, they (1) did not explicitly model new words, or (2) did not give a seamless combination with dis- criminative classifiers (e.g., they just used a threshold to discriminate between known and unknown words). In contrast, unsupervised word segmentation methods (Goldwater et al., 2006; Mochihashi et al., 2009) use nonparametric Bayesian generative models for word generation to infer the “words” only from observations of raw input strings. These methods work quite well and have been used not only for tokenization but also for machine transla- tion (Nguyen et al., 2010), speech recognition (Lee and Glass, 2012; Heymann et al., 2014), and even robotics (Nakamura et al., 2014). However, from a practical point of view, such purely unsupervised approaches do not suffice. Since they only aim to maximize the probability of the language model on the observed set of strings, they sometimes yield word segmentations that are ���¦�·������������������^���º��� ����•�� �Þ��—�������!á��������ñ��×b�� •i•i•�bi�����bº·i��������� ���������ü���������1�� ���p��'������������í� ���ˆ������ Figure …

Content truncated to 500 words