Multimodal Graph-based Transformer Framework for Biomedical Relation Extraction

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3741–3747 August 1–6, 2021. ©2021 Association for Computational Linguistics 3741 Multimodal Graph-based Transformer Framework for Biomedical Relation Extraction Sriram Pingali, Shweta Yadav∗, Pratik Dutta∗, Sriparna Saha Indian Institute of Technology Patna, India {1801cs37, shweta.pcs14, pratik.pcs16, sriparna}@iitp.ac.in Abstract The recent advancement of pre-trained Trans- former models has propelled the development of effective text mining models across various biomedical tasks. However, these models are primarily learned on the textual data and often lack the domain knowledge of the entities to capture the context beyond the sentence. In this study, we introduced a novel framework that enables the model to learn multi-omnics biological information about entities (proteins) with the help of additional multi-modal cues like molecular structure. Towards this, rather developing modality-specific architectures, we devise a generalized and optimized graph based multi-modal learning mechanism that utilizes the GraphBERT model to encode the textual and molecular structure information and exploit the underlying features of various modalities to enable the end-to-end learning. We evaluated our proposed method on Protein- Protein Interaction task from the biomedical corpus, where our proposed generalized ap- proach is observed to be benefited by the ad- ditional domain-specific modality. 1 Introduction The biomedical scientific articles hold the valuable knowledge of biomedical entities (such as protein, drug, gene) and their relationships. However, with the exponential increase in the volume of biomedi- cal articles (Lu, 2011), it is imperative to advance the development of an accurate biomedical text mining tool to extract and curate meaningful infor- mation from huge unstructured texts automatically. One of the cardinal tasks in biomedical doc- ument processing is Protein-protein interaction (PPI), where the relation (‘interaction’ or ‘non- interaction’) between two protein mentions is iden- tified from the given biomedical text. The knowl- ∗These authors contributed equally to this work. edge about protein interactions is critical in under- standing the biological processes, such as signaling cascades, translations and metabolism, that are reg- ulated by the interactions of proteins that alter pro- teins to modulate their stability (Elangovan et al., 2020). Majority of the existing works on PPI in the literature primarily focused only on the textual in- formation present in the biomedical article. How- ever, these approaches lack in capturing (1) multi- omnics biological information regarding protein interactions, and (2) genetic and structure informa- tion of the proteins. A few works (Dutta and Saha, 2020; Asada et al., 2018; Jha et al., 2020; Jha and Saha) have been reported in the literature where the researchers have considered different modalities of the biomedical corpus. However, these multi- modal architectures are modality-specific and thus are very complex. Hence, there is a surge to de- velop a generalized and optimized model that can understand all the modalities rather than develop- ing various architectures for different modalities. Towards this, we explore Graph-based Trans- former model (GraphBERT) (Zhang et al., 2020) to learn the modality independent graph represen- tation. This enables the model to acquire the joint knowledge of both the modalities (textual and …

Content truncated to 500 words