Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction?

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2450–2467 July 9-14, 2023 ©2023 Association for Computational Linguistics Can NLI Provide Proper Indirect Supervision for Low-resource Biomedical Relation Extraction? Jiashu Xu Mingyu Derek Ma Muhao Chen Harvard University University of California, Los Angeles University of Southern California jxu1@harvard.edu ma@cs.ucla.edu muhaoche@usc.edu Abstract Two key obstacles in biomedical relation ex- traction (RE) are the scarcity of annotations and the prevalence of instances without ex- plicitly pre-defined labels due to low anno- tation coverage. Existing approaches, which treat biomedical RE as a multi-class classifica- tion task, often result in poor generalization in low-resource settings and do not have the abil- ity to make selective predictions on unknown cases but give a guess from seen relations, hin- dering the applicability of those approaches. We present NBR, which converts biomedical RE as a natural language inference formula- tion to provide indirect supervision. By con- verting relations to natural language hypothe- ses, NBR is capable of exploiting semantic cues to alleviate annotation scarcity. By in- corporating a ranking-based loss that implic- itly calibrates abstinent instances, NBR learns a clearer decision boundary and is instructed to abstain on uncertain instances. Extensive experiments on three widely-used biomedical RE benchmarks, namely ChemProt, DDI, and GAD, verify the effectiveness of NBR in both full-shot and low-resource regimes. Our analy- sis demonstrates that indirect supervision ben- efits biomedical RE even when a domain gap exists, and combining NLI knowledge with biomedical knowledge leads to the best perfor- mance gains.1 1 Introduction In silico studies of biology and medicine have pri- marily relied on machines’ understanding of rela- tions between various molecules and biomolecules. For instance, disease-target prediction requires ac- curate identification of the association between the drug target and the disease (Bravo et al., 2015), and drug-drug interaction recognition is essential for polypharmacy side effect studies (Herrero-Zazo et al., 2013). Due to the complexity and high cost 1Code is released at https://github.com/luka-group/ NLI_as_Indirect_Supervision of human curation of such biomedical knowledge (Krallinger et al., 2017; Bravo et al., 2015), there has been a growing interest in the field of biomedi- cal relation extraction (RE), a task of automatically inferring the relations between biomedical entities described in domain-specific corpora. However, two obstacles remain in training a reli- able biomedical RE model. First, biomedical RE often suffers from insufficient and imperfect anno- tations, due to that the annotation process is very challenging and requires expert annotators to iden- tify complex structures from lengthy and sophisti- cated biomedical literature. The existing biomed- ical learning resources either require very costly expert annotations (Krallinger et al., 2017) or resort to weak supervision (Bravo et al., 2015). The insuf- ficiency and imperfection of annotations inevitably cause existing state-of-the-art (SOTA) biomedical RE systems (Yasunaga et al., 2022; Peng et al., 2019; Tinn et al., 2021, inter alia), though showing satisfactory results in a fully supervised setting, to result in poor generalization regarding the more common low-resource regime in this domain. For …

Content truncated to 500 words