Unsupervised Domain Adaptation based on Text Relatedness

Proceedings of Recent Advances in Natural Language Processing, pages 733–739, Hissar, Bulgaria, 12-14 September 2011. Unsupervised Domain Adaptation based on Text Relatedness Georgios Petasis Software and Knowledge Engineering Laboratory, Institute of Informatics and Telecommunications, National Centre for Scientific Research “Demokritos”, Athens, Greece petasis@iit.demokritos.gr Abstract In this paper an unsupervised approach to do- main adaptation is presented, which exploits external knowledge sources in order to port a classification model into a new thematic do- main. Our approach extracts a new feature set from documents of the target domain, and tries to align the new features to the original ones, by exploiting text relatedness from external knowledge sources, such as WordNet. The ap- proach has been evaluated on the task of document classification, involving the classifi- cation of newsgroup postings into 20 news groups. 1 Introduction The portability of natural language processing (NLP) systems to new thematic domains is still a research area that attracts a significant research interest. During the last two decades, the use of machine learning has greatly improved the adaptability to new domains, or even languages. However, the vast majority of machine learning algorithms operate under a basic assumption: both the training and test data should use the same feature space, and follow the same distribu- tion, suggesting that both should originate from the same thematic domain. When the distribution changes, the models must be re-generated from newly collected data. The adaptation can be separated into three large categories, according to the available data from the new domain. In supervised approaches, there is an adequate number of labelled data to train the model from scratch, on the new domain. When a limited number of labelled data are available, usually too few to train a model with satisfactory perform- ance, along with unlabeled ones, the adaptation process is characterised as semi-supervised. Fi- nally, unsupervised approaches must adapt their model to a new domain by learning solely from unlabelled examples. Transfer learning or knowledge transfer is a research area, which tries to extract knowledge from previous experience and apply it on new learning tasks. Based on the idea that prior knowledge (i.e. identifying oranges) can be used on new tasks (i.e. identifying lemons), transfer learning researches three main central problems (Zhang and Shakya, 2009): 1) how to extract the prior knowledge that is related, 2) how to repre- sent the knowledge, and 3) how to apply the knowledge in the new learning task. Domain ad- aptation is a sub-category of transfer learning, where (Pan and Yang, 2010): 1. The source and target domains are differ- ent, but related. 2. The source and target tasks are the same (i.e. classification or regression). 3. Labelled examples are available for the source domain. 4. Only unlabeled examples are available for the target domain. In this paper, we propose a novel approach for the task of domain adaptation. Our method con- centrates on the feature space, by trying to ex- pand the features of the source domain with fea- tures that appear only in …

Content truncated to 500 words