Opinion Mining with Deep Recurrent Neural Networks

Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 720–728, October 25-29, 2014, Doha, Qatar. c⃝2014 Association for Computational Linguistics Opinion Mining with Deep Recurrent Neural Networks Ozan ˙Irsoy and Claire Cardie Department of Computer Science Cornell University Ithaca, NY, 14853, USA oirsoy, cardie@cs.cornell.edu Abstract Recurrent neural networks (RNNs) are con- nectionist models of sequential data that are naturally applicable to the analysis of natural language. Recently, “depth in space” — as an orthogonal notion to “depth in time” — in RNNs has been investigated by stacking mul- tiple layers of RNNs and shown empirically to bring a temporal hierarchy to the architec- ture. In this work we apply these deep RNNs to the task of opinion expression extraction formulated as a token-level sequence-labeling task. Experimental results show that deep, narrow RNNs outperform traditional shallow, wide RNNs with the same number of parame- ters. Furthermore, our approach outperforms previous CRF-based baselines, including the state-of-the-art semi-Markov CRF model, and does so without access to the powerful opinion lexicons and syntactic features relied upon by the semi-CRF, as well as without the standard layer-by-layer pre-training typically required of RNN architectures. 1 Introduction Fine-grained opinion analysis aims to detect the sub- jective expressions in a text (e.g. “hate”) and to char- acterize their intensity (e.g. strong) and sentiment (e.g. negative) as well as to identify the opinion holder (the entity expressing the opinion) and the target, or topic, of the opinion (i.e. what the opinion is about) (Wiebe et al., 2005). Fine-grained opinion analysis is important for a variety of NLP tasks including opinion-oriented question answering and opinion summarization. As a result, it has been studied extensively in recent years. In this work, we focus on the detection of opinion ex- pressions — both direct subjective expressions (DSEs) and expressive subjective expressions (ESEs) as de- fined in Wiebe et al. (2005). DSEs consist of explicit mentions of private states or speech events expressing private states; and ESEs consist of expressions that in- dicate sentiment, emotion, etc., without explicitly con- veying them. An example sentence shown in Table 1 in which the DSE “has refused to make any statements” explicitly expresses an opinion holder’s attitude and the The committee , as usual , has O O O B ESE I ESE O B DSE refused to make any statements . I DSE I DSE I DSE I DSE I DSE O Table 1: An example sentence with labels ESE “as usual” indirectly expresses the attitude of the writer. Opinion extraction has often been tackled as a se- quence labeling problem in previous work (e.g. Choi et al. (2005)). This approach views a sentence as a sequence of tokens labeled using the conventional BIO tagging scheme: B indicates the beginning of an opinion-related expression, I is used for tokens inside the opinion-related expression, and O indicates tokens outside any opinion-related class. The example sen- tence in Table 1 shows the appropriate tags in the BIO scheme. For instance, the ESE …

Content truncated to 500 words