Multi-Task Transfer Learning for Weakly-Supervised Relation Extraction

Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, pages 1012–1020, Suntec, Singapore, 2-7 August 2009. c⃝2009 ACL and AFNLP Multi-Task Transfer Learning for Weakly-Supervised Relation Extraction Jing Jiang School of Information Systems Singapore Management University 80 Stamford Road, Singapore 178902 jingjiang@smu.edu.sg Abstract Creating labeled training data for rela- tion extraction is expensive. In this pa- per, we study relation extraction in a spe- cial weakly-supervised setting when we have only a few seed instances of the tar- get relation type we want to extract but we also have a large amount of labeled instances of other relation types. Ob- serving that different relation types can share certain common structures, we pro- pose to use a multi-task learning method coupled with human guidance to address this weakly-supervised relation extraction problem. The proposed framework mod- els the commonality among different re- lation types through a shared weight vec- tor, enables knowledge learned from the auxiliary relation types to be transferred to the target relation type, and allows easy control of the tradeoff between precision and recall. Empirical evaluation on the ACE 2004 data set shows that the pro- posed method substantially improves over two baseline methods. 1 Introduction Relation extraction is the task of detecting and characterizing semantic relations between entities from free text. Recent work on relation extraction has shown that supervised machine learning cou- pled with intelligent feature engineering or ker- nel design provides state-of-the-art solutions to the problem (Culotta and Sorensen, 2004; Zhou et al., 2005; Bunescu and Mooney, 2005; Qian et al., 2008). However, supervised learning heavily re- lies on a sufficient amount of labeled data for train- ing, which is not always available in practice due to the labor-intensive nature of human annotation. This problem is especially serious for relation ex- traction because the types of relations to be ex- tracted are highly dependent on the application do- main. For example, when working in the financial domain we may be interested in the employment relation, but when moving to the terrorism domain we now may be interested in the ethnic and ide- ology affiliation relation, and thus have to create training data for the new relation type. However, is the old training data really useless? Inspired by recent work on transfer learning and domain adaptation, in this paper, we study how we can leverage labeled data of some old relation types to help the extraction of a new relation type in a weakly-supervised setting, where only a few seed instances of the new relation type are avail- able. While transfer learning was proposed more than a decade ago (Thrun, 1996; Caruana, 1997), its application in natural language processing is still a relatively new territory (Blitzer et al., 2006; Daume III, 2007; Jiang and Zhai, 2007a; Arnold et al., 2008; Dredze and Crammer, 2008), and its ap- plication in relation extraction is still unexplored. Our idea of performing transfer learning is mo- tivated by the observation that different relation types …

Content truncated to 500 words