BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis

Proceedings of NAACL-HLT 2019, pages 2324–2335 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2324 BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis Hu Xu1, Bing Liu1, Lei Shu1 and Philip S. Yu1,2 1Department of Computer Science, University of Illinois at Chicago, Chicago, IL, USA 2Institute for Data Science, Tsinghua University, Beijing, China {hxu48, liub, lshu3, psyu}@uic.edu Abstract Question-answering plays an important role in e-commerce as it allows potential customers to actively seek crucial information about prod- ucts or services to help their purchase decision making. Inspired by the recent success of ma- chine reading comprehension (MRC) on for- mal documents, this paper explores the poten- tial of turning customer reviews into a large source of knowledge that can be exploited to answer user questions. We call this problem Review Reading Comprehension (RRC). To the best of our knowledge, no existing work has been done on RRC. In this work, we first build an RRC dataset called ReviewRC based on a popular benchmark for aspect- based sentiment analysis. Since ReviewRC has limited training examples for RRC (and also for aspect-based sentiment analysis), we then explore a novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC. To show the generality of the approach, the proposed post-training is also applied to some other review-based tasks such as aspect ex- traction and aspect sentiment classification in aspect-based sentiment analysis. Experimen- tal results demonstrate that the proposed post- training is highly effective1. 1 Introduction For online commerce, question-answering (QA) serves either as a standalone application of cus- tomer service or as a crucial component of a dia- logue system that answers user questions. Many intelligent personal assistants (such as Amazon Alexa and Google Assistant) support online shop- ping by allowing the user to speak directly to the assistants. One major hindrance for this mode of shopping is that such systems have limited capa- bility to answer user questions about products (or 1The datasets and code are available at https://www. cs.uic.edu/˜hxu/. services), which are vital for customer decision making. As such, an intelligent agent that can automatically answer customers’ questions is very important for the success of online businesses. Given the ever-changing environment of prod- ucts and services, it is very hard, if not impossible, to pre-compile an up-to-date and reliable knowl- edge base to cover a wide assortment of ques- tions that customers may ask, such as in factoid- based KB-QA (Xu et al., 2016; Fader et al., 2014; Kwok et al., 2001; Yin et al., 2015). As a compro- mise, many online businesses leverage community question-answering (CQA) (McAuley and Yang, 2016) to crowdsource answers from existing cus- tomers. However, the problem with this approach is that many questions are not answered, and if they are answered, the answers are delayed, which is not suitable for interactive QA. In this paper, we explore the potential of using product reviews as a large source of user experiences that …

Content truncated to 500 words