Active Learning for the Identification of Nonliteral Language

Proceedings of the Workshop on Computational Approaches to Figurative Language, pages 21–28, Rochester, NY, April 26, 2007. c⃝2007 Association for Computational Linguistics Active Learning for the Identification of Nonliteral Language ∗ Julia Birke and Anoop Sarkar School of Computing Science, Simon Fraser University Burnaby, BC, V5A 1S6, Canada jbirke@alumni.sfu.ca, anoop@cs.sfu.ca Abstract In this paper we present an active learn- ing approach used to create an annotated corpus of literal and nonliteral usages of verbs. The model uses nearly unsu- pervised word-sense disambiguation and clustering techniques. We report on exper- iments in which a human expert is asked to correct system predictions in different stages of learning: (i) after the last iter- ation when the clustering step has con- verged, or (ii) during each iteration of the clustering algorithm. The model obtains an f-score of 53.8% on a dataset in which literal/nonliteral usages of 25 verbs were annotated by human experts. In compari- son, the same model augmented with ac- tive learning obtains 64.91%. We also measure the number of examples required when model confidence is used to select examples for human correction as com- pared to random selection. The results of this active learning system have been com- piled into a freely available annotated cor- pus of literal/nonliteral usage of verbs in context. 1 Introduction In this paper, we propose a largely automated method for creating an annotated corpus of literal vs. nonliteral usages of verbs. For example, given the verb “pour”, we would expect our method to iden- tify the sentence “Custom demands that cognac be poured from a freshly opened bottle” as literal, and the sentence “Salsa and rap music pour out of the windows” as nonliteral, which, indeed, it does. ∗This research was partially supported by NSERC, Canada (RGPIN: 264905). We would like to thank Bill Dolan, Fred Popowich, Dan Fass, Katja Markert, Yudong Liu, and the anonymous reviewers for their comments. We reduce the problem of nonliteral language recognition to one of word-sense disambiguation (WSD) by redefining literal and nonliteral as two different senses of the same word, and we adapt an existing similarity-based word-sense disambigua- tion method to the task of separating usages of verbs into literal and nonliteral clusters. Note that treat- ing this task as similar to WSD only means that we use features from the local context around the verb to identify it as either literal or non-literal. It does not mean that we can use a classifier trained on WSD annotated corpora to solve this issue, or use any existing WSD classification technique that re- lies on supervised learning. We do not have any an- notated data to train such a classifier, and indeed our work is focused on building such a dataset. Indeed our work aims to first discover reliable seed data and then bootstrap a literal/nonliteral identification model. Also, we cannot use any semi-supervised learning algorithm for WSD which relies on reliably annotated seed data since we do not possess any reli- ably labeled data (except for our test …

Content truncated to 500 words