Unsupervised Relation Extraction: A Variational Autoencoder Approach

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1929–1938 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 1929 Unsupervised Relation Extraction: A Variational Autoencoder Approach Chenhan Yuan and Hoda Eldardiry Department of Computer Science, Virginia Tech chenhan,hdardiry@vt.edu Abstract Unsupervised relation extraction works by clustering entity pairs that have the same re- lations in the text. Some existing variational autoencoder (VAE)-based approaches train the relation extraction model as an encoder that generates relation classifications. A decoder is trained along with the encoder to recon- struct the encoder input based on the encoder- generated relation classifications. These clas- sifications are a latent variable so they are re- quired to follow a pre-defined prior distribu- tion which results in unstable training. We pro- pose a VAE-based unsupervised relation ex- traction technique that overcomes this limita- tion by using the classifications as an inter- mediate variable instead of a latent variable. Specifically, classifications are conditioned on sentence input, while the latent variable is con- ditioned on both the classifications and the sen- tence input. This allows our model to connect the decoder with the encoder without putting restrictions on the classification distribution; which improves training stability. Our ap- proach is evaluated on the NYT dataset and outperforms state-of-the-art methods. 1 Introduction Relation Extraction (RE) methods aim to classify relations of entity pairs in a given sentence. RE is the basis of many NLP tasks, such as Informa- tion Extraction (Niklaus et al., 2018), Knowledge Graph Construction (Bosselut et al., 2019) and In- formation Retrieval (Liu et al., 2014). Conven- tional RE models classify relations based on pre- set rules (Zhou et al., 2005; Etzioni et al., 2008). These rules include syntactic patterns such as de- pendency structures and part of speech (POS). Al- though these rule-based methods achieve high ac- curacy, the rules can only be applied to limited types of sentences, which leads to application limi- tations (Wu et al., 2017). Recently, with the rise of deep learning, some work introduced deep neural networks, such as Piece-wise Convolutional Neural Network (PCNN) (Zeng et al., 2015), Graph Con- volutional Networks (Guo et al., 2019), and Graph LSTM (Peng et al., 2017). These approaches per- form well. However, they use supervised model training which requires large-scale labeled data which is not always available. To reduce the reliance on labeled data, some work propose unsupervised relation extraction methods. Traditional unsupervised RE approaches compose vectors based on extracted sentence fea- tures. These feature vectors represent the relation information of sentences (Lin and Pantel, 2001; Yao et al., 2012). These methods then cluster sen- tences according to these feature vectors to classify sentences into relations. Even though these ap- proaches achieve state-of-the-art performance, they heavily rely on hand-craft features and make many simplifying assumptions as mentioned in previous work (Marcheggiani and Titov, 2016). Other un- supervised RE techniques follow a discriminative approach to train more expressive models. These techniques can provide a sufficient learning signal to train deep neural networks as relation classifiers. …

Content truncated to 500 words