Evaluating Multilanguage-Comparability of Subjectivity Analysis Systems

Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 595–603, Uppsala, Sweden, 11-16 July 2010. c⃝2010 Association for Computational Linguistics Evaluating Multilanguage-Comparability of Subjectivity Analysis Systems Jungi Kim, Jin-Ji Li and Jong-Hyeok Lee Division of Electrical and Computer Engineering Pohang University of Science and Technology, Pohang, Republic of Korea {yangpa,ljj,jhlee}@postech.ac.kr Abstract Subjectivity analysis is a rapidly grow- ing field of study. Along with its ap- plications to various NLP tasks, much work have put efforts into multilingual subjectivity learning from existing re- sources. Multilingual subjectivity analy- sis requires language-independent crite- ria for comparable outcomes across lan- guages. This paper proposes to mea- sure the multilanguage-comparability of subjectivity analysis tools, and provides meaningful comparisons of multilingual subjectivity analysis from various points of view. 1 Introduction The field of NLP has seen a recent surge in the amount of research on subjectivity analysis. Along with its applications to various NLP tasks, there have been efforts made to extend the resources and tools created for the English language to other languages. These endeavors have been success- ful in constructing lexicons, annotated corpora, and tools for subjectivity analysis in multiple lan- guages. There are multilingual subjectivity analysis sys- tems available that have been built to monitor and analyze various concerns and opinions on the In- ternet; among the better known are OASYS from the University of Maryland that analyzes opinions on topics from news article searches in multiple languages (Cesarano et al., 2007)1 and TextMap, an entity search engine developed by Stony Brook University for sentiment analysis along with other functionalities (Bautin et al., 2008).2 Though these systems currently rely on English analysis tools and a machine translation (MT) technology to 1http://oasys.umiacs.umd.edu/oasysnew/ 2http://www.textmap.com/ translate other languages into English, up-to-date research provides various ways to analyze subjec- tivity in multilingual environments. Given sentiment analysis systems in differ- ent languages, there are many situations when the analysis outcomes need to be multilanguage- comparable. For example, it has been common these days for the Internet users across the world to share their views and opinions on various top- ics including music, books, movies, and global af- fairs and incidents, and also multinational compa- nies such as Apple and Samsung need to analyze customer feedbacks for their products and services from many countries in different languages. Gov- ernments may also be interested in monitoring ter- rorist web forums or its global reputation. Sur- veying these opinions and sentiments in various languages involves merging the analysis outcomes into a single database, thereby objectively compar- ing the result across languages. If there exists an ideal subjectivity analy- sis system for each language, evaluating the multilanguage-comparability would be unneces- sary because the analysis in each language would correctly identify the exact meanings of all in- put texts regardless of the language. However, this requirement is not fulfilled with current technol- ogy, thus the need for defining and measuring the multilanguage-comparability of subjectivity anal- ysis systems is evident. This paper proposes to evaluate the multilanguage-comparability of multilingual subjectivity …

Content truncated to 500 words