Multi-Task Learning in Conditional Random Fields for Chunking in Shallow Semantic Parsing

Multi-Task Learning in Conditional Random Fields for Chunking in Shallow Semantic Parsing Saike He a, Xiaojie Wang a, Yuan Dong a,b, Taozheng Zhang a, and Xue Bai a aBejing University of Posts and Telecommunications, Beijing, 100876, P.R.China hsk000@gmai1.com, xjwang@bupt.edu.cn, bc003@sina.com bFrance Telecom R&D Center (Beijing), Beijing, 10080, P.R.China yuandong@orange-ftgroup.com Abstract. Alternating Structure Optimization (ASO) is a recently proposed linear Multi- task Learning algorithm. Although its effective has been verified in both semi-supervised as well as supervised methods, yet they necessitate taking external resource as a prerequisite. Therefore, feasibility of employing ASO to further improve the performance merely rests on the labeled data on hand proves to be a task deserving close scrutiny. Catering to this challenging while untapped problem, this paper presents a novel application of ASO to the subtask of Shallow Semantic Parsing: Chunking. Our experiments on Chinese Treebank 5.0 present promising result in chunk analysis, and the error rate is reduced by 5.72%, proposing a profound way to further improve the performance. Keywords: Shallow Semantic Parsing, Chunking, CRFs, Multi-Task Learning, ASO. 1 Introduction Semantic parsing of text corpora is needed to support tasks such as information extraction and question-answering. In particular, shallow semantic parsing focuses on identifying the semantic roles of the arguments of a verb (or any predicate) rather than parsing the whole sentence in detail. Traditional shallow semantic parsing systems for chunk analysis(Kudoh and Matsumoto, 2001; Zhan etal., 2002), which employ machine learning method, focus on the selection of features and their variety combinations to train a statistical model (Carreras and Marquez, 2005). However, we usually have no idea of what a good model is like, neither do we know which features to select. Fortunately, Multi-task Learning (MTL) provides us with a substantial solution to work out this nontrivial problem(Caruana, 1997; Ben-David and Schuller, 2003; Evgeniou and Pontil, 2004; Miccheli and Pontil, 2005; Maurer, 2006). Multi-task Learning is an approach to inductive transfer that emphasizes learning multiple tasks in parallel while using a shared representation, so that what is learned by all tasks is available to the target task. Alternating Structure Optimization (ASO) algorithm (Ando and Zhang, 2005a) is a linear method based on such idea with the shared representation projected as a low-dimensional pro- jection matrix shared by all related tasks. In supervised learning applications, large amount of unlabeled data is readily available, while labeled data are costly to obtain. Therefore, (Ando and Zhang, 2005b) employs ASO in semi-supervised method and seeks to discover shared predictive structures through jointly learning multiple classification problems on unlabeled data. As adequate unlabeled data is involved, promising result achieved. Based on the consideration of huge resource consumption in (Ando and Zhang, 2005b) and the availability of large human-labeled corpora such as PropBank (Palmer et al., 2005) and FrameNet (Baker et al., 1998), (Liu and Ng, 2007) utilizes labeled data to train ASO classifier, reasonably promising performance is obtained with comparably less additional labeled corpora. Copyright 2009 by Saike He, Xiaojie Wang, Yuan Dong, Taozheng Zhang, and Xue Bai 180 …

Content truncated to 500 words