Nearly-Unsupervised Hashcode Representations for Biomedical Relation Extraction

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4026–4036, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 4026 Nearly-Unsupervised Hashcode Representations for Biomedical Relation Extraction Sahil Garg USC ISI sahilgar@usc.edu Aram Galstyan USC ISI galstyan@isi.edu Greg Ver Steeg USC ISI gregv@isi.edu Guillermo Cecchi IBM Research, NY gcecchi@us.ibm.com Abstract Recently, kernelized locality sensitive hash- codes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks. In this paper, we propose to optimize the hashcode representations in a nearly unsu- pervised manner, in which we only use data points, but not their class labels, for learn- ing. The optimized hashcode representa- tions are then fed to a supervised classifier following the prior work. This nearly unsu- pervised approach allows fine-grained op- timization of each hash function, which is particularly suitable for building hashcode representations generalizing from a train- ing set to a test set. We empirically evalu- ate the proposed approach for biomedical relation extraction tasks, obtaining signif- icant accuracy improvements w.r.t. state- of-the-art supervised and semi-supervised approaches. 1 Introduction In natural language processing, one important but a highly challenging task is of identify- ing biological entities and their relations from biomedical text, as illustrated in Fig. 1, rele- vant for real world problems, such as personal- ized cancer treatments (Rzhetsky, 2016; Hahn and Surdeanu, 2015; Cohen, 2015). In the pre- vious works, the task of biomedical relation extraction is formulated as of binary classifi- cation of natural language structures; one of the primary challenges to solve the problem is that the number of data points annotated with class labels (in a training set) is small due to high cost of annotations by biomedi- cal domain experts, and further, bio-text sen- tences in a test set can vary significantly w.r.t. the ones from a training set due to practi- cal aspects, such as high diversity of research topics or writing styles, hedging, etc. Con- sidering such challenges for the task, many classification models based on kernel similar- ity functions have been proposed (Garg et al., 2016; Chang et al., 2016; Tikk et al., 2010; Miwa et al., 2009; Airola et al., 2008; Mooney and Bunescu, 2005), and recently, many neural networks based classification models have also been explored (Kavuluru et al., 2017; Peng and Lu, 2017; Hsieh et al., 2017; Rao et al., 2017; Nguyen and Grishman, 2015), including the ones doing adversarial learning using the knowledge of data points (excluding class la- bels) from a test set, or semi-supervised vari- ational autoencoders (Rios et al., 2018; Ganin et al., 2016; Zhang and Lu, 2019; Kingma et al., 2014). In a very recent work, kernelized local- ity sensitive hashcodes based representation learning approach has been proposed that has shown to be the most successful in terms of accuracy and computational efficiency for the task (Garg et al., 2019). The model param- eters, shared between all …

Content truncated to 500 words