The New Propbank: Aligning Propbank with AMR through POS Unification

The New Propbank: Aligning Propbank with AMR through POS Unification Tim O’Gorman1, Sameer Pradhan23, Julia Bonn1, Katie Conger1, James Gung1,4, Martha Palmer1 1University of Colorado 2cemantix.org 3Vassar College 4IPSoft ogormant@colorado.edu Abstract We present a corpus which converts the sense labels of existing Propbank resources to a new unified format which is more compatible with AMR and more robust to sparsity. This adopts an innovation of the Abstract Meaning Representation project (Banarescu et al., 2013) in which one abstracts away from different, related parts of speech, so that related forms such as “insert” and “insertion” could be represented by the same roleset and use the same semantic roles. We note that this conversion also serves to make the different English Propbank corpora released over the years consistent with each other, so that one might train and evaluate systems upon that larger combined data. We present analysis of some appealing characteristics of this final dataset, and present preliminary results of training and evaluating SRL systems on this combined set, to spur usage of this challenging new dataset. Keywords: Propbank,SRL, Semantic Roles, Corpora 1. Introduction We introduce the conversion of all existing Propbank data — constituting more than half a million predicate instances in English — into a format in which etymologically re- lated senses from different parts are speech are merged, making that data compatible with the predicate senses used for Abstract Meaning Representation (Banarescu et al., 2013). This constitutes a large set of data made consis- tent to use the same frames and conventions, both increas- ing the amount of training data available for Propbank SRL and also providing a large corpus of semantic role label- ing whose rolesets and numbered arguments match those of the Abstract Meaning Representation data (while con- taining over twice as many predicate instances). We describe the combination of automatic and manual methods used in converting these corpora, provide some analysis to characterize the resulting corpora, and present preliminary SRL results against the test sets presented here, as a baseline for future evaluation of SRL. We suggest that evaluating against the combined test sets (OntoNotes, En- glish Web Treebank and BOLT) can provide a challenging test set that could encourage the community to build SRL systems with a greater coverage over nominal, adjectival and light verb data, and with robustness to a range of diffi- cult genres. 1.1. Motivations for Unifying Parts of Speech Propbank (Kingsbury and Palmer, 2002) is a paradigm for the development of semantic role labeling corpora, de- signed for large-scale annotation. It focuses upon an- notation of coarse-grained senses (“rolesets”) which pro- vide predicate-specific definitions of numbered arguments (ARG0, ARG1, etc.) to represent semantic roles. By using coarse-grained sense labels and these predicate- specific arguments (following the “individual thematic roles” of Dowty (1991)), Propbank approaches can achieve a high inter-annotator agreement rate, and the methodol- ogy has been adapted to Chinese (Palmer et al., 2005), Korean (Palmer et al., 2006), Hindi/Urdu (Bhatt et al., 2009), Finnish (Haverinen et al., 2013), Turkish (Sahin, …

Content truncated to 500 words