ABSA-Bench: Towards the Unified Evaluation of Aspect-based Sentiment Analysis Research

ABSA-Bench: Towards the Unified Evaluation of Aspect-based Sentiment Analysis Research Abhishek Das School of Computer Science The University of Adelaide abhishek.das@student.adelaide.edu.au Wei Emma Zhang School of Computer Science The University of Adelaide wei.e.zhang@adelaide.edu.au Abstract Aspect-Based Sentiment Analysis (ABSA) has gained much attention in recent years. ABSA is the task of identifying fine-grained opinion polarity towards a specific aspect as- sociated with a given target. However, there is a lack of benchmarking platform to provide a unified environment under consistent evalu- ation criteria for ABSA, resulting in the dif- ficulties for fair comparisons. In this work, we address this issue and define a benchmark, ABSA-Bench1, by unifying the evaluation pro- tocols and the pre-processed public datasets in a Web-based platform. ABSA-Bench pro- vides two means of evaluations for participants to submit their predictions or models for on- line evaluation. Performances are ranked in the leader board and a discussion forum is sup- ported to serve as a collaborative platform for academics and researchers to discuss queries. 1 Introduction Aspect-based sentiment analysis (ABSA) has gained a lot of attention in recent years from both industries and academic communities as it pro- vides a more practical solution to real life prob- lems. The goal of ABSA is to identify the aspects and infer the sentiment expressed for each aspect. For example, given a sentence I hated their ser- vice, but their food was great, the sentiment polar- ities for the aspect service and food are negative and positive respectively. Conventional techniques for ABSA are mostly traditional machine learning models based on lexicons and syntactic features (Jiang et al., 2011; Kiritchenko et al., 2014; Vo and Zhang, 2015). Therefore, the performance of such models depend on hand-crafted features. Recent progresses have been made with the advancement of Deep Neural Networks (DNN) with some of the models being considered as state-of-the-art (Xu 1https://absa-bench.com/ Figure 1: The General Process of ABSA et al.). Among them, attention mechanism has played an important role outperforming previous approaches by paying more attention to the con- text words that are semantically-closer with the as- pect terms (Luong et al., 2015; Wang et al., 2016; Chen et al., 2017; Liu et al., 2018; Ma et al., 2017). The most recent approaches adopted pre-trained Bidirectional Encoder Representations from Trans- formers (BERT) architecture (Devlin et al., 2019; Xu et al.) generating significant performance gaps to other approaches due to BERT’s capability of capturing bi-directional contextual information and providing rich token-wise representation. Introduc- ing BERT architecture into ABSA task naturally distinguishing the approaches to Non-BERT based models and BERT-based models. Figure 1 depicts the general processes of both of the two groups of supervised ABSA methods. Although this research area has gained much attention in recent years, it lacks of unbiased com- parisons overall. As deep learning based models perform differently on various hardware on differ- ent deep learning tools, existing works typically chose to either re-run or re-implement the selected comparative models under their own experimental environment. We also observe few works directly …

Content truncated to 500 words