Figure Me Out: A Gold Standard Dataset for Metaphor Interpretation

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 5810–5819 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 5810 Figure Me Out: A Gold Standard Dataset for Metaphor Interpretation Omnia Zayed, John P. McCrae, Paul Buitelaar Insight Centre for Data Analytics Data Science Institute National University of Ireland Galway IDA Business Park, Lower Dangan, Galway, Ireland {omnia.zayed, john.mccrae, paul.buitelaar}@insight-centre.org Abstract Metaphor comprehension and understanding is a complex cognitive task that requires interpreting metaphors by grasping the interaction between the meaning of their target and source concepts. This is very challenging for humans, let alone computers. Thus, automatic metaphor interpretation is understudied in part due to the lack of publicly available datasets. The creation and manual annotation of such datasets is a demanding task which requires huge cognitive effort and time. Moreover, there will always be a question of accuracy and consistency of the annotated data due to the subjective nature of the problem. This work addresses these issues by presenting an annotation scheme to interpret verb-noun metaphoric expressions in text. The proposed approach is designed with the goal of reducing the workload on annotators and maintain consistency. Our methodology employs an automatic retrieval approach which utilises external lexical resources, word embeddings and semantic similarity to generate possible interpretations of identified metaphors in order to enable quick and accurate annotation. We validate our proposed approach by annotating around 1,500 metaphors in tweets which were annotated by six native English speakers. As a result of this work, we publish as linked data the first gold standard dataset for metaphor interpretation which will facilitate research in this area. Keywords: metaphor, interpretation, dataset, annotation, tweets, lexical resources. 1. Introduction Metaphor is a crucial aspect of human cognition and com- munication. The computational processing of metaphors has gained wide attention lately by focusing on two tasks, namely, metaphor identification and metaphor interpreta- tion. Metaphor identification is concerned with recognis- ing the metaphoric word or expression in a given sentence while metaphor interpretation focuses on “translating” the metaphor to its literal meaning. The interpretation task is very important to fully understand the intended meaning of the metaphor, however it is much less explored compared to the identification task. One reason is that it is very exhaust- ing for humans to comprehend the interaction between the target and the source components of the metaphoric expres- sion. Although native speakers unconsciously grasp such interaction, asking a human annotator to translate such a cognitive process and interpret a metaphoric expression is a very demanding task. This is the reason behind the lack of publicly available datasets for metaphor interpretation, which in turn hinders the development of this topic. There are several approaches to address metaphor interpre- tation among which: 1. Lexical Substitution (lexical paraphrasing) where the metaphoric word/phrase is replaced with its literal counterpart to clarify its semantic meaning. This task is viewed as single-word (lexical) substitu- tion (Shutova, 2010; Shutova et al., 2012; Bollegala and Shutova, 2013); 2. Paraphrase …

Content truncated to 500 words