Model Transfer with Explicit Knowledge of the Relation between Class Definitions

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018), pages 541–550 Brussels, Belgium, October 31 - November 1, 2018. c⃝2018 Association for Computational Linguistics 541 Model Transfer with Explicit Knowledge of the Relation between Class Definitions Hiyori Yoshikawa and Tomoya Iwakura Fujitsu Laboratories Ltd., Kanagawa, Japan {y.hiyori, iwakura.tomoya}@jp.fujitsu.com Abstract This paper investigates learning methods for multi-class classification using labeled data for the target classification scheme and another la- beled data for a similar but different classi- fication scheme (support scheme). We show that if we have prior knowledge about the re- lation between support and target classification schemes in the form of a class correspondence table, we can use it to improve the model per- formance further than the simple multi-task learning approach. Instead of learning the in- dividual classification layers for the support and target schemes, the proposed method con- verts the class label of each example on the support scheme into a set of candidate class labels on the target scheme via the class cor- respondence table, and then uses the candi- date labels to learn the classification layer for the target scheme. We evaluate the proposed method on two tasks in NLP. The experimental results show that our method effectively learns the target schemes especially for the classes that have a tight connection to certain support classes. 1 Introduction Machine learning based methods have shown high performance in many NLP tasks, which typically are formulated as some kinds of classification problems. Although there has been a remarkable progress in methods utilizing unlabeled resources, many tasks still require a large amount (at least thousands, in some cases millions or billions) of high quality labeled data to achieve high accuracy. For many tasks, however, classification schemes vary depending on fields of application or other factors, and large and high quality labeled data following a single scheme is insufficient. Named entity recognition (NER) (Nadeau and Sekine, 2007) and text classification (TC) (Joachims, 1998) are typical examples that allow variable classification schemes. For example, there are two kinds of NE type definitions for Japanese NER: IREX (Sekine and Isahara, 2000) with eight entity types and Sekine’s extended NE (ENE) hierarchy with 200 entity types (Sekine et al., 2002). These two schemes are relevant but not in complete correspondence, i.e., a class in ENE is not necessarily a proper subclass of a class in IREX and vice versa. For example, entities with LOCATION type in IREX are (a subtype of) LOCATION or FACILITY in ENE, while some entities with FACILITY type in ENE can also be ORGANIZATION in IREX. It is also the case that a classification scheme for an existing model is revised. In the case of news categolization, for example, a new category such as world cup would be added at a certain point of time; or the articles about eSports would newly categorised the existing sports category. To obtain labeled data following the desired scheme, it is often required to create them al- most from scratch …

Content truncated to 500 words