BERT4GCN: Using BERT Intermediate Layers to Augment GCN for Aspect-based Sentiment Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9193–9200 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9193 BERT4GCN: Using BERT Intermediate Layers to Augment GCN for Aspect-based Sentiment Classification Zeguan Xiao1, Jiarun Wu1, Qingliang Chen1,2∗, Congjian Deng3 1Department of Computer Science, Jinan University, Guangzhou 510632, China 2Guangzhou Xuanyuan Research Institute Company, Ltd., Guangzhou 510006, China 3Guangzhou Yunqu Information Technology Company, Ltd., Guangzhou 510665, China xiaozg@stu2018.jnu.edu.cn, benwu@stu2019.edu.cn tpchen@jnu.edu.cn, dcj@yunqu-info.com Abstract Graph-based Aspect-based Sentiment Classifi- cation (ABSC) approaches have yielded state- of-the-art results, expecially when equipped with contextual word embedding from pre- training language models (PLMs). However, they ignore sequential features of the context and have not yet made the best of PLMs. In this paper, we propose a novel model, BERT4GCN, which integrates the grammati- cal sequential features from the PLM of BERT, and the syntactic knowledge from dependency graphs. BERT4GCN utilizes outputs from in- termediate layers of BERT and positional in- formation between words to augment GCN (Graph Convolutional Network) to better en- code the dependency graphs for the down- stream classification. Experimental results demonstrate that the proposed BERT4GCN outperforms all state-of-the-art baselines, justi- fying that augmenting GCN with the grammat- ical features from intermediate layers of BERT can significantly empower ABSC models. 1 Introduction Aspect-based sentiment classification (ABSC), a fine-grained sentiment classification task in the field of sentiment analysis, aims at identifying the sentiment polarities of aspects explicitly given in sentences. For example, given the sentence “The price is reasonable although the service is poor.”, ABSC needs to correctly assign a positive polarity to price and a negative one to service. Intuitively, matching aspects with their corre- sponding opinion expressions is the core of ABSC task. Some of previous deep learning approaches (Tang et al., 2016; Ma et al., 2017; Huang et al., 2018; Song et al., 2019) use various types of atten- tion mechanisms to model the relationship between aspects and opinion expressions in an implicit way. However, these attention-based models do not take ∗ Corresponding author (Qingliang Chen). good advantage of the syntactic information of sen- tences, such as the dependency graph, to better associate the aspects with their sentimental polari- ties, thus leading to poor performance. In order to better pair aspects and their corre- sponding opinion expressions, the syntactic fea- tures must be taken into account. Recent work such as (Huang and Carley, 2019; Sun et al., 2019; Zhang et al., 2019; Wang et al., 2020; Tang et al., 2020) employs either Graph Convolutional Networks (GCNs) (Kipf and Welling, 2017) or Graph Attention Networks (GATs) (Velickovic et al., 2018) on the dependency graph, in which words in the sentence are nodes in the graph and grammatical relations are edge labels. The depen- dency graph they use can be obtained from either ordinary dependency tree parser or reshaping it using heuristic rules. These models can achieve promisingly better performance with this additional grammatical features, especially when they incor- porate the contextual word embedding, such as BERT (Devlin et al., 2019). …

Content truncated to 500 words