Case Study of Model Adaptation: Transfer Learning and Online Learning

International Joint Conference on Natural Language Processing, pages 1292–1298, Nagoya, Japan, 14-18 October 2013. Case Study of Model Adaptation: Transfer Learning and Online Learning Kenji Imamura NTT Media Intelligence Laboratories 1-1 Hikari-no-oka, Yokosuka, 239-0847 Japan imamura.kenji@lab.ntt.co.jp Abstract Many NLP tools are released as programs that include statistical models. Unfortu- nately, the models do not always match the documents that the tool user is inter- ested in, which forces the user to update the models. In this paper, we investigate model adap- tation under the condition that users can- not access the data used in creating the original model. Transfer learning and on- line learning are investigated as adaptation strategies. We test them on the category classification of Japanese newspaper arti- cles. Experiments show that both trans- fer and online learning can appropriately adapt the original model if the dataset for adaptation contains all data, not just the data that cannot be well handled by the original model. In contrast, we confirmed that the adaptation fails if the dataset con- tains only erroneous data as indicated by the original model. 1 Introduction Recent natural language processing (NLP) sys- tems are built using machine learning (supervised learning). The developers of these systems basi- cally create annotated corpora from which statis- tical models are generated. However, if the docu- ments that users want to apply the systems to do not belong to the domain of the annotated corpora, the resulting accuracy tends to be unsatisfactory. For instance, Figure 1 shows the typical drop in accuracy in the category classification task of newspaper articles over time; the statistical model was trained using supervised data from 1995 (de- tails are described later). Even though the test data were obtained from newspaper articles (i.e., the same domain data), the accuracy against 2007 60% 65% 70% 75% Accuracy 55% 60% 65% 70% 75% 1995 2000 2005 Accuracy Year Figure 1: Accuracy of Category Classification with training by 1995 Dataset articles fell by about 10% from the 1996 articles. The reasons for this include the emergence of new words and changes in word distribution. In order to recover this degradation, we have to re-train the models. To overcome this problem, transfer learning methods have been proposed (Pan and Yang, 2010). Many transfer learning methods assume that the users can obtain both the original data and additional data for adaptation. However, in most practical cases, the users sometimes are unable to access the original data. For example, only the de- velopers are licensed to handle the original data, not the users. NLP tools, such as taggers, parsers, and clas- sifiers, are commonly released as programs that include the original models. Since many users cannot update the original models, they continue to use them even if the user’s documents do not match the models (Figure 2). The objective of this paper is to investigate methods that, given an additional dataset, permit adaptation of original models under the constraint that the original dataset is unavailable. The target task of this paper …

Content truncated to 500 words