A Corpus of Metaphor Novelty Scores for Syntactically-Related Word Pairs

A Corpus of Metaphor Novelty Scores for Syntactically-Related Word Pairs Natalie Parde and Rodney D. Nielsen Department of Computer Science and Engineering University of North Texas {natalie.parde, rodney.nielsen}@unt.edu Abstract Automatically scoring metaphor novelty is an unexplored topic in natural language processing, and research in this area could benefit a wide range of NLP tasks. However, no publicly available metaphor novelty datasets currently exist, making it difficult to perform research on this topic. We introduce a large corpus of metaphor novelty scores for syntactically related word pairs, and release it freely to the research community. We describe the corpus here, and include an analysis of its score distribution and the types of word pairs included in the corpus. We also provide a brief overview of standard metaphor detection corpora, to provide the reader with greater context regarding how this corpus compares to other datasets used for different types of computational metaphor processing. Finally, we establish a performance benchmark to which future researchers can compare, and show that it is possible to learn to score metaphor novelty on our dataset at a rate ignificantly better than chance or na¨ıve strategies. Keywords: metaphor, metaphor novelty, figurative language, crowdsourcing 1. Introduction Metaphors exist along a novelty continuum, with highly conventional metaphors (those that a person is likely to en- counter on a day-to-day basis, e.g., “I spent an hour on my homework.”) at one end, and highly novel metaphors (those that stand out as being particularly uncommon or creative, e.g., “She frowned like a thunderstorm.”) at the other, with many falling somewhere in between. Despite this, research on computational metaphor processing to date has largely viewed metaphor as a binary phenomenon. This simplifi- cation has at least partially arisen due to a lack of metaphor novelty-annotated data, as pointed out by Haagsma and Bjerva (2016). Resource scarcity notwithstanding, having a reliable means of automatically scoring metaphor novelty could be of ben- efit to many tasks. Word sense disambiguation, for exam- ple, can be quite adept at handling conventional metaphors (simply treating them as additional word senses), but novel metaphors require more complex processing (Shutova, 2015; Haagsma and Bjerva, 2016). Automatically grad- ing essays, asessing text difficulty levels, and identifying interesting topics for book discussion systems could all benefit from an ability to automatically score metaphors. Metaphor novelty can also be leveraged for psychologi- cal and cognitive health assessment. For example, Gutier- rez et al. (2017) applied standard, binary metaphor detec- tion to open-ended patient interviews to aid in the detec- tion of schizophrenia, motivated by clinical research in- dicating that schizophrenic patients tend to produce par- ticularly bizarre figurative speech (Kuperberg, 2010; Bil- low et al., 1997). Although the binary metaphor labels were useful, one can hypothesize that performance could be further magnified with continuous novelty scores. Like- wise, research in cognitive neuroscience has found that pa- tients with Alzheimer’s disease often struggle with compre- hending novel metaphors but not conventional metaphors (Amanzio et al., 2008). Systems capable of distinguishing between different grades of …

Content truncated to 500 words