Modular Self-Supervision for Document-Level Relation Extraction

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5291–5302 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5291 Modular Self-Supervision for Document-Level Relation Extraction Sheng Zhang1 Cliff Wong1 Naoto Usuyama1 Sarthak Jain2∗ Tristan Naumann1 Hoifung Poon1 1Microsoft Research, Redmond, WA 2Khoury College of Computer Sciences, Northeastern University {shezhan,clwon,naotous,tristan,hoifung}@microsoft.com jain.sar@husky.neu.edu Abstract Extracting relations across large text spans has been relatively underexplored in NLP, but it is particularly important for high-value domains such as biomedicine, where obtaining high re- call of the latest findings is crucial for practical applications. Compared to conventional infor- mation extraction confined to short text spans, document-level relation extraction faces addi- tional challenges in both inference and learn- ing. Given longer text spans, state-of-the-art neural architectures are less effective and task- specific self-supervision such as distant super- vision becomes very noisy. In this paper, we propose decomposing document-level relation extraction into relation detection and argument resolution, taking inspiration from Davidso- nian semantics. This enables us to incorporate explicit discourse modeling and leverage mod- ular self-supervision for each sub-problem, which is less noise-prone and can be further refined end-to-end via variational EM. We con- duct a thorough evaluation in biomedical ma- chine reading for precision oncology, where cross-paragraph relation mentions are preva- lent. Our method outperforms prior state of the art, such as multi-scale learning and graph neural networks, by over 20 absolute F1 points. The gain is particularly pronounced among the most challenging relation instances whose ar- guments never co-occur in a paragraph. 1 Introduction Prior work on information extraction tends to fo- cus on binary relations within sentences. However, practical applications often require extracting com- plex relations across large text spans. This is es- pecially important in high-value domains such as biomedicine, where obtaining high recall of the latest findings is crucial. For example, Figure 1 shows a ternary (drug, gene, mutation) relation signifying that a tumor with MAP2K1 mutation ∗Work done as an intern at Microsoft Research. "... The patient’s peripheral blood indices are shown over time relative to the first dose of the MEK inhibitor cobimetinib ..." (... 17 sentences spanning 2 paragraphs ...) "... MAP2K1 mutations appeared later with p.K57T expanding to become the dominant clone ..." (... 10 sentences spanning 3 paragraphs ...) "... Expression of the MAP2K1 mutants in a lymphoid cell line showed that while each mutation was able to activate ERK in the presence of vemurafenib, all mutations remained sensitive to MEK inhibitors ..." Figure 1: An example document-level drug-gene- mutation relation. cobimetinib never co-occurs with MAP2K1 or K57T in any paragraph. K57T is sensitive to cobimetinib, yet the entities never co-occur in any single paragraph. Such pre- cision oncology knowledge is key for determining personalized treatment for cancer patients, but it is scattered among a vast biomedical literature of more than 30 million papers, with over 1 million being added each year1. Recently, there has been increasing interest in cross-sentence relation extraction, but most exist- ing work still focuses on short text spans. Quirk …

Content truncated to 500 words