From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5539–5548, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5539 From the Token to the Review: A Hierarchical Multimodal approach to Opinion Mining Alexandre Garcia, Pierre Colombo, Slim Essid, Florence d’Alché-Buc, Chloé Clavel Télécom ParisTech Université Paris Saclay {garcia,pcolombo,sessid,fdalche,cclavel}@telecom-paristech.fr Abstract The task of predicting fine grained user opin- ion based on spontaneous spoken language is a key problem arising in the development of Computational Agents as well as in the devel- opment of social network based opinion min- ers. Unfortunately, gathering reliable data on which a model can be trained is notoriously dif- ficult and existing works rely only on coarsely labeled opinions. In this work we aim at bridg- ing the gap separating fine grained opinion models already developed for written language and coarse grained models developed for spon- taneous multimodal opinion mining. We take advantage of the implicit hierarchical structure of opinions to build a joint fine and coarse grained opinion model that exploits different views of the opinion expression. The resulting model shares some properties with attention- based models and is shown to provide compet- itive results on a recently released multimodal fine grained annotated corpus. 1 Introduction Recent years have witnessed the increasing popu- larity of social networks and video streaming plat- forms. People heavily rely on these channels to express their opinions through video-based discus- sions or reviews. Whereas such opinionated data has been widely studied in the context of written customer reviews (Liu, 2012) crawled on websites such as Amazon (Hu and Liu, 2004) and IMDB (Maas et al., 2011), only a few studies have been proposed in the case of video-based reviews. Such multimodal data has been shown to provide a mean to disambiguate some hard to understand opinion expressions such as irony and sarcasm (Attardo et al., 2003) and contains crucial information in- dicating the level of engagement and the persua- siveness of the speaker (Clavel and Callejas, 2016; Ben Youssef et al., 2019; Nojavanasghari et al., 2016). A key problem in this context is the lack of availability of fine grained opinion annotation i.e. annotations performed at the token or short span level and highlighting on the components of the structure of opinions. Indeed whereas such re- sources have been gathered in the case of textual data and can be used to deeply understand the ex- pression of opinions (Wiebe et al., 2005; Pontiki et al., 2016), the different attempts at annotating multimodal reviews have shown that reaching good annotator agreement is nearly impossible at a fine grained level. This results from the disfluent aspect of spontaneous spoken language making it difficult to choose opinions’ annotation boundaries (Garcia et al., 2019; Langlet and Clavel, 2015b). Thus the price to pay to gather reliable data is the definition of an annotation scheme focusing on coarse grained information such as long segment categorization as done by Zadeh …

Content truncated to 500 words