Spatial Multi-Arrangement for Clustering and Multi-way Similarity Dataset Construction

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5749–5758 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5749 Spatial Multi-Arrangement for Clustering and Multi-way Similarity Dataset Construction Olga Majewska1, Diana McCarthy1, Jasper van den Bosch2, Nikolaus Kriegeskorte3, Ivan Vuli´c1, Anna Korhonen1 1 University of Cambridge, Dept. of Theoretical and Applied Linguistics, Cambridge, CB3 9DA, United Kingdom 2 University of Birmingham, School of Psychology, Edgbaston, Birmingham, B15 2TT, United Kingdom 3 Columbia University, Jerome L. Greene Science Center, 3227 Broadway, New York, NY 10027, United States {om304, iv250, alk23}@cam.ac.uk, diana@dianamccarthy.co.uk, vandejjf@bham.ac.uk, nk2765@columbia.edu Abstract We present a novel methodology for fast bottom-up creation of large-scale semantic similarity resources to support development and evaluation of NLP systems. Our work targets verb similarity, but the methodology is equally applicable to other parts of speech. Our approach circumvents the bottleneck of slow and expensive manual development of lexical resources by leveraging semantic intuitions of native speakers and adapting a spatial multi-arrangement approach from cognitive neuroscience, used before only with visual stimuli, to lexical stimuli. Our approach critically obtains judgments of word similarity in the context of a set of related words, rather than of word pairs in isolation. We also handle lexical ambiguity as a natural consequence of a two-phase process where verbs are placed in broad semantic classes prior to the fine-grained spatial similarity judgments. Our proposed design produces a large-scale verb resource comprising 17 relatedness-based classes and a verb similarity dataset containing similarity scores for 29,721 unique verb pairs and 825 target verbs, which we release with this paper. Keywords: Lexicon, Lexical Database, Semantics, Crowdsourcing 1. Introduction Verbs pose a particular challenge to machine interpretation of sentence meaning due to their complex linguistic proper- ties. The role as sentence pivots, encoding crucial informa- tion about the structural and semantic relationships between the elements of the clause, assigns a lot of weight to verbs as carriers of information. This is why accurate, nuanced analysis and representation of their meaning is especially important for NLP systems to get closer to human levels of language understanding (Jackendoff, 1972; Levin, 1993; McRae et al., 1997; Altmann and Kamide, 1999; Resnik and Diab, 2000; Ferretti et al., 2001; Sauppe, 2016, inter alia). The demand for verb-specific resources to support NLP has been recognised in recent years, as reflected in the publica- tion of a large verb similarity dataset for English, SimVerb- 3500 (hereafter SimVerb) (Gerz et al., 2016). However, the need for high-quality, wide-coverage lexical resources targeting verb semantics has by no means been satisfied. Rich lexical resources encoding information about verbs’ semantic properties such as FrameNet (Baker et al., 1998) or VerbNet (Kipper Schuler, 2005; Kipper et al., 2006) are still unavailable for most languages, and evaluation datasets dedicated to or dominated by nouns are by far predominant (Finkelstein et al., 2002; Agirre et al., 2009; Bruni et al., 2012; Hill et al., 2015). Therefore, we propose methodology aimed at alleviating the evaluation data scarcity problem and …

Content truncated to 500 words