SemEval-2010 Task 7: Argument Selection and Coercion

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 27–32, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics SemEval-2010 Task 7: Argument Selection and Coercion James Pustejovsky and Anna Rumshisky and Alex Plotnick Dept. of Computer Science Brandeis University Waltham, MA, USA Elisabetta Jezek Dept. of Linguistics University of Pavia Pavia, Italy Olga Batiukova Dept. of Humanities Carlos III University of Madrid Madrid, Spain Valeria Quochi ILC-CNR Pisa, Italy Abstract We describe the Argument Selection and Coercion task for the SemEval-2010 eval- uation exercise. This task involves char- acterizing the type of compositional oper- ation that exists between a predicate and the arguments it selects. Specifically, the goal is to identify whether the type that a verb selects is satisfied directly by the argument, or whether the argument must change type to satisfy the verb typing. We discuss the problem in detail, describe the data preparation for the task, and analyze the results of the submissions. 1 Introduction In recent years, a number of annotation schemes that encode semantic information have been de- veloped and used to produce data sets for training machine learning algorithms. Semantic markup schemes that have focused on annotating entity types and, more generally, word senses, have been extended to include semantic relationships between sentence elements, such as the seman- tic role (or label) assigned to the argument by the predicate (Palmer et al., 2005; Ruppenhofer et al., 2006; Kipper, 2005; Burchardt et al., 2006; Subi- rats, 2004). In this task, we take this one step further and attempt to capture the “compositional history” of the argument selection relative to the predicate. In particular, this task attempts to identify the oper- ations of type adjustment induced by a predicate over its arguments when they do not match its se- lectional properties. The task is defined as fol- lows: for each argument of a predicate, identify whether the entity in that argument position satis- fies the type expected by the predicate. If not, then identify how the entity in that position satisfies the typing expected by the predicate; that is, identify the source and target types in a type-shifting or co- ercion operation. Consider the example below, where the verb re- port normally selects for a human in subject po- sition, as in (1a). Notice, however, that through a metonymic interpretation, this constraint can be violated, as demonstrated in (1b). (1) a. John reported in late from Washington. b. Washington reported in late. Neither the surface annotation of entity extents and types nor assigning semantic roles associated with the predicate would reflect in this case a cru- cial point: namely, that in order for the typing requirements of the predicate to be satisfied, a type coercion or a metonymy (Hobbs et al., 1993; Pustejovsky, 1991; Nunberg, 1979; Egg, 2005) has taken place. The SemEval Metonymy task (Markert and Nis- sim, 2007) was a good attempt to annotate such metonymic relations over a larger data set. This task involved two types with their …

Content truncated to 500 words