Predicting Humorousness and Metaphor Novelty with Gaussian Process Preference Learning

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5716–5728 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 5716 Predicting Humorousness and Metaphor Novelty with Gaussian Process Preference Learning Edwin Simpson* and Erik-Lân Do Dinh* and Tristan Miller*† and Iryna Gurevych* *Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universität Darmstadt https://www.ukp.tu-darmstadt.de/ †Austrian Research Institute for Artificial Intelligence (OFAI) Freyung 6, 1010 Vienna, Austria http://www.ofai.at/ Abstract The inability to quantify key aspects of creat- ive language is a frequent obstacle to natural language understanding. To address this, we introduce novel tasks for evaluating the cre- ativeness of language—namely, scoring and ranking text by humorousness and metaphor novelty. To sidestep the difficulty of assigning discrete labels ornumeric scores, we learnfrom pairwise comparisons between texts. We intro- duce a Bayesian approach for predicting humor- ousness and metaphor novelty using Gaussian process preference learning (GPPL), which achieves a Spearman’s ρ of 0.56 against gold using word embeddings and linguistic features. Our experimentsshow that given sparse, crowd- sourced annotation data, ranking using GPPL outperforms best–worst scaling. We release a new dataset for evaluating humour contain- ing 28,210 pairwise comparisons of 4030 texts, and make our software freely available. 1 Introduction Creative language, such as humour and metaphor, is an essential part of everyday communication, yet remains a challenge for computational meth- ods. Unlike much literal language, humour and figurative language require complex linguistic and background knowledge to understand, which are dif- ficult to integrate with NLP methods (Hempelmann, 2008; Shutova, 2010). An important step in processing creative lan- guage is to recognise its presence in a piece of text. Humour and metaphors are two of the most fre- quently used types of creative language whose use most obscures the true meaning of a piece of text from its surface interpretation (Raskin, 1985, pp. 1– 5, 100–104; Black, 1955) and whose attributes, such as funniness and novelty, may be present or per- ceived to varying degrees (Bell, 2017; Dunn, 2010). For example, the level of appreciation (i.e., humor- ousness or equivalently funniness) of jokes can vary according to their content and structural features, such as nonsense or disparagement (Carretero-Dios et al., 2010) or, in the case of puns, contextual coher- ence (Lippman and Dunn, 2000) and the cognitive effort required to recover the target word (Hempel- mann, 2003, pp. 123–124). With metaphors, the literal meaning of frequently used metaphors can drop out of everyday usage, leaving the metaphorical sense as the expected one (Shutova, 2015). For such conventionalised metaphors, NLP methods may identify the meta- phorical sense from training data or resources such as WordNet, whereas novel metaphors require the ability to recognise the analogy being made. While previous work (see §2) has considered mainly binary classification approaches to humour or metaphor recognition, this paper focuses on quantifying humorousness and metaphor novelty. These tasks are important for downstream applic- ations such as conversational agents or machine translation, which must choose the correct tone in …

Content truncated to 500 words