Improved relation classification by deep recurrent neural networks with data augmentation

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1461–1470, Osaka, Japan, December 11-17 2016. Improved Relation Classification by Deep Recurrent Neural Networks with Data Augmentation Yan Xu,1,∗,‡ Ran Jia,1,∗ Lili Mou,1 Ge Li,1,† Yunchuan Chen,2 Yangyang Lu,1 Zhi Jin1,† 1Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education, China; Institute of Software, Peking University {xuyan14,lige,luyy11,zhijin}@sei.pku.edu.cn {jiaran1994,doublepower.mou}gmail.com 2University of Chinese Academy of Sciences chenyunchuan11@mails.ucas.ac.cn Abstract Nowadays, neural networks play an important role in the task of relation classification. By de- signing different neural architectures, researchers have improved the performance to a large ex- tent in comparison with traditional methods. However, existing neural networks for relation classification are usually of shallow architectures (e.g., one-layer convolutional neural networks or recurrent networks). They may fail to explore the potential representation space in different abstraction levels. In this paper, we propose deep recurrent neural networks (DRNNs) for rela- tion classification to tackle this challenge. Further, we propose a data augmentation method by leveraging the directionality of relations. We evaluated our DRNNs on the SemEval-2010 Task 8, and achieve an F1-score of 86.1%, outperforming previous state-of-the-art recorded results.1 1 Introduction Classifying relations between two entities in a given context is an important task in natural language pro- cessing (NLP). Take the following sentence as an example: “Jewelry and other smaller [valuables]e1 were locked in a [safe]e2 or a closet with a deadbolt.” The marked entities valuables and safe are of relation Content-Container(e1, e2). Relation classification plays a key role in various NLP applications, and has become a hot research topic in recent years. Nowadays, neural network-based approaches have made significant improvement in relation classifi- cation, compared with traditional methods based on either human-designed features (Kambhatla, 2004; Hendrickx et al., 2009) or kernels (Bunescu and Mooney, 2005; Plank and Moschitti, 2013). For exam- ple, Zeng et al. (2014) and Xu et al. (2015a) utilize convolutional neural networks (CNNs) for relation classification. Xu et al. (2015b) apply long short term memory (LSTM)-based recurrent neural networks (RNNs) along the shortest dependency path. Nguyen and Grishman (2015) build ensembles of gated recurrent unit (GRU)-based RNNs and CNNs. We have noticed that these neural models are typically designed in shallow architectures, e.g., one layer of CNN or RNN, whereas evidence in the deep learning community suggests that deep architectures are more capable of information integration and abstraction (Graves et al., 2013; Hermans and Schrauwen, 2013; Irsoy and Cardie, 2014). A natural question is then whether such deep architectures are beneficial to the relation classification task. In this paper, we propose the deep recurrent neural networks (DRNNs) to classify relations. The deep RNNs can explore the representation space in different levels of abstraction and granularity. By visualizing how RNN units are related to the ultimate classification, we demonstrate that different layers indeed learn different representations: low-level layers enable sufficient information mix, while high- level layers are more capable of precisely locating the information relevant to the target relation between ∗Equal contribution. †Corresponding authors. …

Content truncated to 500 words