A Multi-lingual Annotated Dataset for Aspect-Oriented Opinion Mining

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2533–2538, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. A Multi-lingual Annotated Dataset for Aspect-Oriented Opinion Mining Salud Mar´ıa Jim´enez Zafra1, Giacomo Berardi2, Andrea Esuli2, Diego Marcheggiani2, Mar´ıa Teresa Mart´ın-Valdivia1, Alejandro Moreo Fern´andez2 1Departamento de Inform´atica, Escuela Polit´ecnica Superior de Ja´en Universidad de Ja´en, E-23071 - Ja´en, Spain {sjzafra, maite}@ujaen.es 2Istituto di Scienza e Tecnologie dell’Informazione “A. Faedo” Consiglio Nazionale delle Ricerche, I-56124 - Pisa, Italy {firstname.lastname}@isti.cnr.it Abstract We present the Trip-MAML dataset, a Multi-Lingual dataset of hotel reviews that have been manually annotated at the sentence-level with Multi-Aspect senti- ment labels. This dataset has been built as an extension of an existent English-only dataset, adding documents written in Ital- ian and Spanish. We detail the dataset construction process, covering the data gathering, selection, and annotation. We present inter-annotator agreement figures and baseline experimental results, compar- ing the three languages. Trip-MAML is a multi-lingual dataset for aspect-oriented opinion mining that enables researchers (i) to face the problem on languages other than English and (ii) to the experiment the application of cross-lingual learning meth- ods to the task. 1 Introduction Reviews of products and services that are sponta- neously produced by customers represent a source of unquestionable value not only for marketing strategies of private companies and organizations, but also for other users since their purchasing de- cisions are likely influenced by other customers’ opinions (Chevalier and Mayzlin, 2006). Overall ratings (e.g., in terms of a five stars rat- ing scale), and also aspect-specific ratings (e.g., the Cleanliness or Location of a hotel), are the typi- cal additional information expressed by customers in their reviews. Those ratings help to derive a number of global scores to facilitate a first screen- ing of the product or service at hand. Notwith- standing, users who pay more attention to a par- ticular aspect (e.g., the Rooms of a hotel) remain constrained to manually inspect the entire text of reviews in order to find out the reasons other users argued in that respect. Methods for au- tomatic analysis of the aspect-oriented sentiment expressed in reviews would enable highlighting aspect-relevant parts of the document, so as to al- low users to perform a faster and focused inspec- tion of them. Previous work on opinion mining (Pang and Lee, 2008) has already faced the overall sentiment prediction (Pang et al., 2002), multiple aspect- oriented analysis (Hu and Liu, 2004), and fine- grained phrase-level analysis (Wilson et al., 2009). Most of the available opinion mining datasets con- tain only documents written in English, as this lan- guage is the most used on the Internet and the one for which more NLP tools and resources are avail- able. (Hu and Liu, 2004) worked on the summa- rization of reviews by means of weakly supervised feature mining. (T¨ackstr¨om and McDonald, 2011) used a finer-grained dataset in which global po- larity annotation is applied also to each sentence composing the document. Similarly did (Socher et al., …

Content truncated to 500 words