VPS-GradeUp: Graded Decisions on Usage Patterns

VPS-GradeUp: Graded Decisions on Usage Patterns V´ıt Baisa♣, Silvie Cinkov´a♠, Ema Krejˇcov´a♠, Anna Vernerov´a♠ ♣Masaryk University, Brno, Faculty of Informatics, NLP Centre ♠Charles University in Prague, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics xbaisa@fi.muni.cz, {cinkova,krejcova,vernerova}@ufal.mff.cuni.cz Abstract We present VPS-GradeUp – a set of 11,400 graded human decisions on usage patterns of 29 English lexical verbs from the Pattern Dictionary of English Verbs by Patrick Hanks. The annotation contains, for each verb lemma, a batch of 50 concordances with the given lemma as KWIC, and for each of these concordances we provide a graded human decision on how well the individual PDEV patterns for this particular lemma illustrate the given concordance, indicated on a 7-item Likert scale for each PDEV pattern. With our annotation, we were pursuing a pilot investigation of the foundations of human clustering and disambiguation decisions with respect to usage patterns of verbs in context. The data set is publicly available at http://hdl.handle.net/11234/1-1585. Keywords: CPA, graded decisions, English, verbs, usage patterns, annotation 1. Introduction We present VPS-GradeUp – a set of 11,400 graded human decisions on usage patterns of 29 English lexical verbs from the Pattern Dictionary of English Verbs1(Hanks, 2000 2014). The annotation contains, for each verb lemma, a batch of 50 concordances with the given lemma as KWIC (key word in context), and for each of these concordances we provide a graded human decision on how well the individual PDEV patterns for this particular lemma illustrate the given concordance, indicated on a 7-item Likert scale for each PDEV pattern. This data set has been created to observe interannotator agreement on PDEV patterns produced using the Corpus Pattern Analysis (Hanks, 2013). The manually annotated concordances of PDEV are sometimes regarded as the gold standard for an intuitively plausible clustering of verb uses, and they have already been used for statistical machine learning. Efforts have been laid into automating either directly the clustering of unseen concordances from scratch or at least their classification according to predefined patterns; e.g. (Popescu, 2013; Baisa et al., 2015) and (Materna, 2013). In this context, we have created a new data set to investigate the foundations of human clustering and disambiguation decisions. VPS-GradeUp is now publicly available at http://hdl.handle.net/11234/1-1585. 2. Related Work VPS-GradeUp draws on previous research associated with the VPS-30-En data set (Cinkov´a et al., 2012). Both data sets are similar in that they provide parallel human annotations of PDEV data and that they process 50-concordance batches for each selected verb entry. Nevertheless, the data sets differ in two important aspects: 1The entry structure of PDEV is presented in Fig. 1. In a nutshell, each category in a PDEV entry contains a pattern (formulated as a finite clause template with argument labels from PDEV’s own ontology) and an implicature – another finite clause template with semantic types, explaining or paraphrasing the meaning of the pattern. 1. While VPS-30-En allows revisions of PDEV entries to optimize the interannotator agreement, VPS-GradeUp sticks to the original PDEV patterns (referring to their …

Content truncated to 500 words