ProGene - A Large-scale, High-Quality Protein-Gene Annotated Benchmark Corpus

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 4585–4596 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 4585 PROGENE — A Large-scale, High-Quality Protein-Gene Annotated Benchmark Corpus Erik Faessler, Luise Modersohn, Christina Lohr and Udo Hahn Jena University Language & Information Engineering (JULIE) Lab, Friedrich-Schiller-Universit¨at Jena, Jena, Germany SMITH Consortium of the German Medical Informatics Initiative {erik.faessler | luise.modersohn | christina.lohr | udo.hahn}@uni-jena.de Abstract Genes and proteins are the fundamental entities of molecular genetics and deeper knowledge about their interactions constitutes a cor- nerstone for advancing precision medicine. We here introduce PROGENE (formerly called FSU-PRGE), a corpus that reflects our efforts to cope with this important class of named entities within the framework of a long-lasting large-scale annotation campaign at the Jena University Language & Information Engineering (JULIE) Lab. We partitioned the entire corpus into 11 subcorpora covering various biological domains to achieve an overall subdomain-independent corpus. It consists of 3,308 MEDLINE abstracts with over 36k sen- tences and more than 960k tokens annotated with nearly 60k named entity mentions. Two annotators strove for carefully assigning entity mentions to classes of genes/proteins as well as families/groups, complexes, variants and enumerations of those where genes and proteins are represented by a single class. The main purpose of the corpus is to provide a large body of consistent and reliable annotations for supervised training and evaluation of machine learning algorithms in this relevant domain. Furthermore, we provide an evaluation of two state-of-the-art baseline systems—BIOBERT and FLAIR—on the PROGENE corpus. We make the evaluation datasets and the trained models available as a benchmark to encourage comparable evaluations of new methods in the future. Keywords: Genes and proteins, text corpus, annotation, biomedical corpus, evaluation 1. Introduction Genes and proteins are the fundamental entities of molecu- lar genetics and deeper knowledge about their interactions constitutes a cornerstone for advancing precision medicine. Hence, they attracted the interest of the biomedical NLP community for a long time already, with focus on two infor- mation extraction tasks. The first one, named entity recog- nition and grounding, aims at locating gene and protein mentions in a document and, in a second step, mapping such mentions onto some well-established identifier system or name space, such as provided by Gene Ontology (GO) (Gene Ontology Consortium, 2001), UNIPROT (UniProt Consortium, 2017), NCBI GENE (Brown et al., 2015) or KEGG (Kanehisa et al., 2012). Both tasks were particu- larly featured in several iterations of the BIOCREATIVE Challenge (Hirschman et al., 2005; Krallinger et al., 2008; Arighi et al., 2011; Mao et al., 2014). The second task aims at extracting relations between genes or proteins in terms gene/protein interactions from documents (as investigated in several iterations within the BioCreAtIvE (Krallinger et al., 2008; Leitner et al., 2010; Arighi et al., 2011; Do˘gan et al., 2017) and BioNLP relation extraction challenges (Kim et al., 2009; Pyysalo et al., 2011; Kim et al., 2011; Pyysalo et al., 2012; Kim et al., 2013; N´edellec et al., 2013; …

Content truncated to 500 words