The Universal Decompositional Semantics Dataset and Decomp Toolkit

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5698–5707 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5698 The Universal Decompositional Semantics Dataset and Decomp Toolkit Aaron Steven White▷, Elias Stengel-Eskin◁, Siddharth Vashishtha▷, Venkata Govindarajan▷◁, Dee Ann Reisinger, Tim Vieira◁, Keisuke Sakaguchi†, Sheng Zhang‡, Francis Ferraro◦, Rachel Rudinger†, Kyle Rawlins◁, Benjamin Van Durme◁ ▷University of Rochester, ◁Johns Hopkins University, ◦University of Maryland Baltimore County †Allen Institute for Artificial Intelligence, ‡Microsoft Research, ▷◁University of Texas at Austin {aaron.white, svashis3}@rochester.edu, {estenge2, kgr, vandurme}@jhu.edu, shezhan@microsoft.com, {keisukes, rudinger}@allenai.org, ferraro@umbc.edu, {gvenkata1994, dee.ann.reisinger, tim.f.vieira}@gmail.com Abstract We present the Universal Decompositional Semantics (UDS) dataset (v1.0), which is bundled with the Decomp toolkit (v0.1). UDS1.0 unifies five high-quality, decompositional semantics-aligned annotation sets within a single semantic graph specification—with graph structures defined by the predicative patterns produced by the PredPatt tool and real-valued node and edge attributes constructed using sophisticated normalization procedures. The Decomp toolkit provides a suite of Python 3 tools for querying UDS graphs using SPARQL. Both UDS1.0 and Decomp0.1 are publicly available at http://decomp.io. Keywords: semantics, semantic roles, factuality, genericity, temporal duration, entity typing 1. Introduction Traditional semantic annotation frameworks generally de- fine complex, often exclusive category systems that require highly trained annotators to build (Palmer et al., 2005; Ba- narescu et al., 2013; Abend and Rappoport, 2013; Oepen et al., 2014; Oepen et al., 2015; Bos et al., 2017; Abzianidze et al., 2017; Abzianidze and Bos, 2017; Schneider et al., 2018). And in spite of their high quality for the cases they are designed to handle, these frameworks can be brittle to cases that (i) deviate from prototypical instances of a cate- gory; (ii) are equally good instances of multiple categories; or (iii) fall under a category that was erroneously excluded from the framework’s ontology.1 An alternative approach to semantic annotation that ad- dresses these issues has been growing in popularity: de- compositional semantics (Reisinger et al., 2015; White et al., 2016). In this approach, which is rooted in a long tradi- tion of theoretical approaches to lexical semantics (Puste- jovsky, 1995; Levin and Rappaport Hovav, 2005, and refer- ences therein), semantic annotation takes the form of many simple questions about words or phrases (in context) that are easy for na¨ıve native speakers to answer, thus allowing annotations to be crowd-sourced while retaining high inter- annotator agreement. The decompositional approach can be thought of as a feature-based counterpart to traditional category-based sys- tems, with each question determining a semantic feature. Common feature configurations often correspond to cate- gories in a traditional framework (Reisinger et al., 2015; Govindarajan et al., 2019); but unlike such frameworks, a decompositional approach retains the ability to capture con- figurations that were not considered at design time. Further, unlike a categorical framework, reannotation after an over- haul of the framework’s ontology is never necessary, since additional annotations simply accrue to sharpen the frame- work’s ability to capture fine-grained semantic phenomena. A variety of semantic annotation datasets that take a decom- 1Shalev …

Content truncated to 500 words