SemEval-2010 Task 18: Disambiguating Sentiment Ambiguous Adjectives

Proceedings of the 5th International Workshop on Semantic Evaluation, ACL 2010, pages 81–85, Uppsala, Sweden, 15-16 July 2010. c⃝2010 Association for Computational Linguistics SemEval-2010 Task 18: Disambiguating Sentiment Ambiguous Adjectives Yunfang Wu Key Laboratory of Computational Linguistics (Peking University)， Ministry of Education, China wuyf@pku.edu.cn Peng Jin Laboratory of Intelligent Information Processing and Application, Leshan Normal University, China jinp@lstc.edu.cn Abstract Sentiment ambiguous adjectives cause major difficulties for existing algorithms of sentiment analysis. We present an evaluation task designed to provide a framework for comparing different approaches in this problem. We define the task, describe the data creation, list the participating systems and discuss their results. There are 8 teams and 16 systems. 1 Introduction In recent years, sentiment analysis has attracted considerable attention (Pang and Lee, 2008). It is the task of mining positive and negative opinions from natural language, which can be applied to many natural language processing tasks, such as document summarization and question answering. Previous work on this problem falls into three groups: opinion mining of documents, sentiment classification of sentences and polarity prediction of words. Sentiment analysis both at document and sentence level rely heavily on word level. The most frequently explored task at word level is to determine the semantic orientation (SO) of words, in which most work centers on assigning a prior polarity to words or word senses in the lexicon out of context. However, for some words, the polarity varies strongly with context, making it hard to attach each to a specific sentiment category in the lexicon. For example, consider “low cost” versus “low salary”. The word “low” has a positive orientation in the first case but a negative orientation in the second case. Turney and Littman (2003) claimed that sentiment ambiguous words could not be avoided easily in a real-world application in the future research. But unfortunately, sentiment ambiguous words are discarded by most research concerning sentiment analysis (Hatzivassiloglou and McKeown, 1997; Turney and Littman, 2003; Kim and Hovy, 2004). The exception work is Ding et al. (2008). They call these words as context dependant opinions and propose holistic lexicon-based approach to solve thisa problem. The language they deal with is English. The disambiguation of sentiment ambiguous words can also be considered as a problem of phrase-level sentiment analysis. Wilson et al. (2005) present a two-step process to recognize contextual polarity that employs machine learning and a variety of features. Takamura et al. (2006, 2007) propose latent variable model and lexical network to determine SO of phrases, focusing on “noun+adjective” pairs. Their experimental results suggest that the classification of pairs containing ambiguous adjectives is much harder than those with unambiguous adjectives. The task 18 at SemEval 2010 provides a benchmark data set to encourage studies on this problem. This paper is organized as follows. Section 2 defines the task. Section 3 describes the data annotation. Section 4 gives a brief summary of 16 participating systems. Finally Section 5 draws conclusions. 2 Task Set up 2.1 Task description In this task, we focus on …

Content truncated to 500 words