An Empirical Study of Active Learning with Support Vector Machines forJapanese Word Segmentation

An Empirical Study of Active Learning with Support Vector Machines for Japanese Word Segmentation Manabu Sassano Fujitsu Laboratories Ltd. 4-1-1, Kamikodanaka, Nakahara-ku, Kawasaki 211-8588, Japan sassano@jp.fujitsu.com Abstract We explore how active learning with Sup- port Vector Machines works well for a non-trivial task in natural language pro- cessing. We use Japanese word segmenta- tion as a test case. In particular, we discuss how the size of a pool affects the learning curve. It is found that in the early stage of training with a larger pool, more la- beled examples are required to achieve a given level of accuracy than those with a smaller pool. In addition, we propose a novel technique to use a large number of unlabeled examples effectively by adding them gradually to a pool. The experimen- tal results show that our technique requires less labeled examples than those with the technique in previous research. To achieve 97.0 % accuracy, the proposed technique needs 59.3 % of labeled examples that are required when using the previous tech- nique and only 17.4 % of labeled exam- ples with random sampling. 1 Introduction Corpus-based supervised learning is now a stan- dard approach to achieve high-performance in nat- ural language processing. However, the weakness of supervised learning approach is to need an anno- tated corpus, the size of which is reasonably large. Even if we have a good supervised-learning method, we cannot get high-performance without an anno- tated corpus. The problem is that corpus annotation is labour intensive and very expensive. In order to overcome this, some unsupervised learning methods and minimally-supervised methods, e.g., (Yarowsky, 1995; Yarowsky and Wicentowski, 2000), have been proposed. However, such methods usually de- pend on tasks or domains and their performance of- ten does not match one with a supervised learning method. Another promising approach is active learning, in which a classifier selects examples to be labeled, and then requests a teacher to label them. It is very dif- ferent from passive learning, in which a classifier gets labeled examples randomly. Active learning is a general framework and does not depend on tasks or domains. It is expected that active learning will reduce considerably manual annotation cost while keeping performance. However, few papers in the field of computational linguistics have focused on this approach (Dagan and Engelson, 1995; Thomp- son et al., 1999; Ngai and Yarowsky, 2000; Hwa, 2000; Banko and Brill, 2001). Although there are many active learning methods with various classi- fiers such as a probabilistic classifier (McCallum and Nigam, 1998), we focus on active learning with Sup- port Vector Machines (SVMs) because of their per- formance. The Support Vector Machine, which is introduced by Vapnik (1995), is a powerful new statistical learn- ing method. Excellent performance is reported in hand-written character recognition, face detection, image classification, and so forth. SVMs have been recently applied to several natural language tasks, including text classification (Joachims, 1998; Dumais et al., 1998), chunking (Kudo and Mat- sumoto, 2000b; Kudo and Matsumoto, 2001), and dependency analysis (Kudo …

Content truncated to 500 words