Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction

Proceedings of NAACL-HLT 2018, pages 872–884 New Orleans, Louisiana, June 1 - 6, 2018. c⃝2018 Association for Computational Linguistics Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction Patrick Verga, Emma Strubell, Andrew McCallum College of Information and Computer Sciences University of Massachusetts Amherst {pat, strubell, mccallum}@cs.umass.edu Abstract Most work in relation extraction forms a prediction by looking at a short span of text within a single sentence containing a single entity pair mention. This approach often does not consider interactions across mentions, requires redundant computation for each mention pair, and ignores rela- tionships expressed across sentence bound- aries. These problems are exacerbated by the document- (rather than sentence-) level annotation common in biological text. In response, we propose a model which simul- taneously predicts relationships between all mention pairs in a document. We form pair- wise predictions over entire paper abstracts using an efficient self-attention encoder. All- pairs mention scores allow us to perform multi-instance learning by aggregating over mentions to form entity pair representa- tions. We further adapt to settings without mention-level annotation by jointly training to predict named entities and adding a cor- pus of weakly labeled data. In experiments on two Biocreative benchmark datasets, we achieve state of the art performance on the Biocreative V Chemical Disease Relation dataset for models without external KB re- sources. We also introduce a new dataset an order of magnitude larger than existing human-annotated biological information ex- traction datasets and more accurate than distantly supervised alternatives. 1 Introduction With few exceptions (Swampillai and Stevenson, 2011; Quirk and Poon, 2017; Peng et al., 2017), nearly all work in relation extraction focuses on clas- sifying a short span of text within a single sentence containing a single entity pair mention. However, relationships between entities are often expressed across sentence boundaries or otherwise require a larger context to disambiguate. For example, 30% of relations in the Biocreative V CDR dataset (§3.1) are expressed across sentence boundaries, such as in the following excerpt expressing a relationship be- tween the chemical azathioprine and the disease fibrosis: Treatment of psoriasis with azathioprine. Azathioprine treatment benefited 19 (66%) out of 29 patients suffering from severe pso- riasis. Haematological complications were not troublesome and results of biochemical liver function tests remained normal. Min- imal cholestasis was seen in two cases and portal fibrosis of a reversible degree in eight. Liver biopsies should be undertaken at regular intervals if azathioprine therapy is contin- ued so that structural liver damage may be detected at an early and reversible stage. Though the entities’ mentions never occur in the same sentence, the above example expresses that the chemical entity azathioprine can cause the side effect fibrosis. Relation extraction models which consider only within-sentence relation pairs can- not extract this fact without knowledge of the complicated coreference relationship between eight and azathioprine treatment, which, without features from a complicated pre-processing pipeline, cannot be learned by a model which considers entity pairs in isolation. Making separate predictions for each mention pair also obstructs …

Content truncated to 500 words