Cross-Domain Evaluation of Edge Detection for Biomedical Event Extraction

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 1982–1989 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 1982 Cross-Domain Evaluation of Edge Detection for Biomedical Event Extraction Alan Ramponi,1,2,3 Barbara Plank,2 Rosario Lombardo3 1 Department of Information Engineering and Computer Science, University of Trento, Italy 2 Department of Computer Science, IT University of Copenhagen, Denmark 3 Fondazione the Microsoft Research – University of Trento Centre for Computational and Systems Biology, Italy alan.ramponi@unitn.it, bplank@itu.dk, lombardo@cosbi.eu Abstract Biomedical event extraction is a crucial task in order to automatically extract information from the increasingly growing body of biomedical literature. Despite advances in the methods in recent years, most event extraction systems are still evaluated in-domain and on complete event structures only. This makes it hard to determine the performance of intermediate stages of the task, such as edge detection, across different corpora. Motivated by these limitations, we present the first cross-domain study of edge detection for biomedical event extraction. We analyze differences between five existing gold standard corpora, create a standardized benchmark corpus, and provide a strong baseline model for edge detection. Experiments show a large drop in performance when the baseline is applied on out-of-domain data, confirming the need for domain adaptation methods for the task. To encourage research efforts in this direction, we make both the data and the baseline available to the research community: https://www.cosbi.eu/cfx/9985. Keywords: Corpus (Creation, Annotation, etc.), Information Extraction, Information Retrieval, Statistical and Machine Learning Methods. 1. Introduction Information extraction systems in the biomedical field are valuable for a wide range of purposes, from the popula- tion of knowledge bases to the construction of biochem- ical pathways (Ananiadou et al., 2010). Among the nat- ural language processing tasks for information extraction is Event Extraction (EE). Its goal is to extract semanti- cally rich, structured information from unstructured texts. These representations, called events, are suitable to capture the elaborate biomedical statements in the scientific litera- ture.1 The expressivity of EE commonly comes at the cost of multiple classification stages (Figure 1). Given the in- put text and entity annotations, these stages are: (1) trig- ger detection: the identification of words – usually verbs or nominalized verbs – that may trigger events, and the assignment of the event type they express; (2) edge de- tection: the identification and classification of the seman- tic relations which hold between trigger words and named entities (or other triggers); and (3) event construction: the building of complete, multi-argument event structures from 1.the edges. For instance, consider the example in Figure Firstly, “phosphorylation” and “augments” are iden- tified as Phosphorylation and +Regulation trig- gers, respectively. Then, arguments for those event triggers are determined (e.g., Phosphorylation is the Cause of +Regulation). Lastly, arguments are composed into self-contained event structures: two Phosphorylation events and two +Regulation events. Recent studies on EE have shown that supervised machine learning approaches and in particular neural methods pro- vide state-of-the-art performance on the task (Bj¨orne and Salakoski, 2018; Li et …

Content truncated to 500 words