Relation Classification via Multi-Level Attention CNNs

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 1298–1307, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Relation Classification via Multi-Level Attention CNNs Linlin Wang1∗, Zhu Cao1∗, Gerard de Melo2, Zhiyuan Liu3† 1Institute for Interdisciplinary Information Sciences, Tsinghua University, Beijing, China 2 Department of Computer Science, Rutgers University, Piscataway, NJ, USA 3State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China {ll-wang13,cao-z13}@mails.tsinghua.edu.cn, gdm@demelo.org Abstract Relation classification is a crucial ingredi- ent in numerous information extraction sys- tems seeking to mine structured facts from text. We propose a novel convolutional neural network architecture for this task, relying on two levels of attention in order to better discern patterns in heterogeneous contexts. This architecture enables end- to-end learning from task-specific labeled data, forgoing the need for external knowl- edge such as explicit dependency structures. Experiments show that our model outper- forms previous state-of-the-art methods, in- cluding those relying on much richer forms of prior knowledge. 1 Introduction Relation classification is the task of identifying the semantic relation holding between two nominal en- tities in text. It is a crucial component in natural language processing systems that need to mine ex- plicit facts from text, e.g. for various information extraction applications as well as for question an- swering and knowledge base completion (Tandon et al., 2011; Chen et al., 2015). For instance, given the example input “Fizzy [drinks] and meat cause heart disease and [diabetes].” with annotated target entity mentions e1 = “drinks” and e2 = “diabetes”, the goal would be to automati- cally recognize that this sentence expresses a cause- effect relationship between e1 and e2, for which we use the notation Cause-Effect(e1,e2). Accurate relation classification facilitates precise sentence interpretations, discourse processing, and higher- level NLP tasks (Hendrickx et al., 2010). Thus, ∗ Equal contribution. † Corresponding author. Email: liuzy@tsinghua.edu.cn relation classification has attracted considerable at- tention from researchers over the course of the past decades (Zhang, 2004; Qian et al., 2009; Rink and Harabagiu, 2010). In the example given above, the verb corre- sponds quite closely to the desired target relation. However, in the wild, we encounter a multitude of different ways of expressing the same kind of relationship. This challenging variability can be lexical, syntactic, or even pragmatic in nature. An effective solution needs to be able to account for useful semantic and syntactic features not only for the meanings of the target entities at the lexical level, but also for their immediate context and for the overall sentence structure. Thus, it is not surprising that numerous feature- and kernel-based approaches have been proposed, many of which rely on a full-fledged NLP stack, including POS tagging, morphological analysis, de- pendency parsing, and occasionally semantic anal- ysis, as well as on knowledge resources to capture lexical and semantic features (Kambhatla, 2004; Zhou et al., 2005; Suchanek et al., 2006; Qian et al., 2008; Mooney and Bunescu, 2005; Bunescu …

Content truncated to 500 words