Semantic Proto-Roles

Semantic Proto-Roles Drew Reisinger Rachel Rudinger Francis Ferraro Craig Harman Kyle Rawlins∗ Benjamin Van Durme∗ {rawlins@cogsci, vandurme@cs}.jhu.edu Johns Hopkins University Abstract We present the first large-scale, corpus based verification of Dowty’s seminal theory of proto-roles. Our results demonstrate both the need for and the feasibility of a property-based annotation scheme of semantic relationships, as opposed to the currently dominant notion of categorical roles. 1 Introduction For decades researchers have debated the number and character of thematic roles required for a theory of the syntax/semantics interface. AGENT and PA- TIENT are canonical examples, but questions emerge such as: should we have a distinct role for BENE- FICIARY? What about RECIPIENT? What are the boundaries between these roles? And so on. Dowty (1991), in a seminal article, responded to this debate by constructing the notion of a Proto- Agent and Proto-Patient, based on entailments that can be mapped to questions, such as: “Did the ar- gument change state?”, or “Did the argument have volitional involvement in the event?”. Dowty argued that these properties group together in the lexicon non-categorically, in a way that aligns with clas- sic Agent/Patient intuitions. For instance, a Proto- Patient often both changes state (but might not), and often is causally affected by another participant. Various resources have been developed for com- putational linguists working on ‘Semantic Role La- beling’ (SRL), largely under the classical, categor- ical notion of role. Here we revisit Dowty’s re- ∗Corresponding authors. search as computational linguists desiring data for a new task, Semantic Proto-Role Labeling (SPRL), in which existing coarse-grained categorical roles are replaced by scalar judgements of Dowty-inspired properties. As the availability of supporting data is a critical component of such a task, much of our ef- forts here are focused on showing that everyday En- glish speakers (untrained annotators) are able to an- swer basic questions about semantic relationships. In this work we consider the following questions: (i) can crowdsourcing methods be used to empiri- cally validate the formal linguistic theory of Dowty, following prior work in psycholinguistics (Kako, 2006b)? (ii) How might existing semantic anno- tation efforts be used in such a pursuit? (iii) Can the pursuit of Dowty’s semantic properties be turned into a practical and scalable annotation task? (iv) Do the results of such an annotation task (at various scales, including over very large corpora) continue to confirm Dowty’s proto- role hypothesis? And fi- nally, (v) how do the resulting configurations of fine- grained role properties compare to coarser annotated roles in resources such as VerbNet?1 We first derive a set of basic semantic ques- tions pertaining to Dowty-inspired properties. These questions are used in two Mechanical Turk HITs that address the above issuess. In the first HIT, we build on psycholinguistic work (Kako, 2006b) to directly access ‘type-level’ intuitions about a lexical item, by asking subjects property-questions using made-up (“nonce”) words in argument positions. Our results 1To be clear, Dowty himself does not make direct predic- tions about the distribution of proto-role properties within a cor- pus, …

Content truncated to 500 words