An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9174–9179 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 9174 An Empirical Study on Leveraging Position Embeddings for Target-oriented Opinion Words Extraction Samuel Mensah Computer Science Department University of Sheffield, UK s.mensah@sheffield.ac.uk Kai Sun BDBC and SKLSDE Beihang University, China sunkai@buaa.edu.cn Nikolaos Aletras Computer Science Department University of Sheffield, UK n.aletras@sheffield.ac.uk Abstract Target-oriented opinion words extraction (TOWE) (Fan et al., 2019b) is a new subtask of target-oriented sentiment analysis that aims to extract opinion words for a given aspect in text. Current state-of-the-art methods leverage position embeddings to capture the relative position of a word to the target. However, the performance of these methods depends on the ability to incorporate this information into word representations. In this paper, we explore a variety of text encoders based on pretrained word embeddings or language models that leverage part-of-speech and position embeddings, aiming to examine the actual contribution of each component in TOWE. We also adapt a graph convolutional network (GCN) to enhance word representa- tions by incorporating syntactic information. Our experimental results demonstrate that BiLSTM-based models can effectively encode position information into word representations while using a GCN only achieves marginal gains. Interestingly, our simple methods outperform several state-of-the-art complex neural structures. 1 Introduction Target-oriented opinion words extraction (TOWE) (Fan et al., 2019b) is a fine-grained task of target- oriented sentiment analysis (Liu, 2012) aiming to extract opinion words with respect to an opinion target (or aspect) in text. Given the sentence “The food is good but the service is extremely slow”, TOWE attempts to identify the opinion words “good” and “extremely slow” corresponding respec- tively to the targets “food” and “service”. TOWE is usually treated as a sequence labeling problem using the BIO tagging scheme (Ramshaw and Mar- cus, 1999) to distinguish the Beginning, Inside and Outside of a span of opinion words. Table 1 shows an example of applying the BIO tagging scheme for TOWE. Sentence: The food is good but the service is extremely slow. True Labels for target ‘food’: The/O food/O is/O good/B but/O the/O service/O is/O extremely/O slow/O. True Labels for target ‘service’: The/O food/O is/O good/O but/O the/O service/O is/O extremely/B slow/I. TOWE Extraction Results: {(food, good), (service, extremely slow)} Table 1: Identifying target-oriented opinion words in a sentence. Underlined words are opinion targets. Spans tagged B and I are considered as opinion words. Learning effective word representations is a crit- ical step towards tackling TOWE. Traditional work (Zhuang et al., 2006a; Hu and Liu, 2004a; Qiu et al., 2011) has used hand-crafted features to represent words which do not often generalize easily. More recent work (Liu et al., 2015; Fan et al., 2019b; Wu et al., 2020a; Veyseh et al., 2020) has explored neural networks to learn word representations auto- matically. Previous neural-based methods (Liu et al., 2015; Fan et al., 2019b) has used word embeddings (Col- lobert and Weston, 2008; Mikolov et al., 2013; Pen- nington et al., 2014) to represent …

Content truncated to 500 words