PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1714–1719 Barcelona, Spain (Online), December 8-13, 2020 1714 POD: Positional Dependency-Based Word Embedding for Aspect Term Extraction Yichun Yin1, Chenguang Wang2, Ming Zhang3∗ 1 Noah’s Ark Lab, Huawei 2 Computer Science Division, UC Berkeley 3 Department of Computer Science, Peking University yinyichun@huawei.com,chenguangwang@berkeley.edu mzhang cs@pku.edu.cn Abstract Dependency context-based word embedding jointly learns the representations of word and depen- dency context, and has been proved effective in aspect term extraction. In this paper, we design the positional dependency-based word embedding (POD) which considers both dependency context and positional context for aspect term extraction. Specifically, the positional context is mod- eled via relative position encoding. Besides, we enhance the dependency context by integrating more lexical information (e.g., POS tags) along dependency paths. Experiments on SemEval 2014/2015/2016 datasets show that our approach outperforms other embedding methods in aspect term extraction. 1 Introduction Aspect term extraction aims to extract expressions that represent properties of products or services from online reviews (Hu and Liu, 2004a; Hu and Liu, 2004b; Popescu and Etzioni, 2007; Liu, 2010). Understanding the context between words in reviews, such as through conditional random fields (Pontiki et al., 2014; Pontiki et al., 2015; Pontiki et al., 2016), is the key to superior results in aspect term extraction. Word embeddings are effective to capture the contextual information across a wide range of NLP tasks (Tai et al., 2015; Lei et al., 2015; Bojanowski et al., 2017; Devlin et al., 2019). However, they only produce moderate results in aspect term extraction. Recent studies (e.g., Yin et al. (2016)) indicate that this is due to the distributed nature of the word embedding (Mikolov et al., 2013b), which ignores the rich context between the words, such as syntactic information. In this paper, we propose positional dependency-based word embedding (POD) to enhance the con- text modeling capability for aspect term extraction. POD explicitly captures two types of contexts, dependency context and positional context. Inspired by the simple-yet-effective position encoding in Transformer (Vaswani et al., 2017), POD models the positional context via relative position encod- ing (Shaw et al., 2018) between words within a fixed window. Besides, the dependency context is defined as the dependency path as well as the attached lexical information (e.g., POS tags and words) along the path. Moreover, POD is able to incorporate more lexical information into the semantic compositional model via the dependency context, making representations of dependency paths more informative than the ones that only consider grammatical information (Yin et al., 2016). We then linearly combine the dependency and positional context to produce the positional dependencies among words. We also define a margin-based ranking loss to efficiently optimize POD. Our contributions are two-fold, (i) we propose positional dependency-based word embedding POD, which incorporates both positional context and dependency context, (ii) we compare POD with existing aspect term extraction methods and demonstrate that POD yields improved results on aspect term extraction datasets. ∗Corresponding author. This work is licensed under a Creative Commons Attribution 4.0 …

Content truncated to 500 words