Document Classification for COVID-19 Literature

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3715–3722 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 3715 Document Classification for COVID-19 Literature Bernal Jim´enez Guti´errez, Juncheng Zeng, Dongdong Zhang, Ping Zhang, Yu Su The Ohio State University {jimenezgutierrez.1,zeng.671,zhang.11069, zhang.10631,su.809}@osu.edu Abstract The global pandemic has made it more im- portant than ever to quickly and accurately re- trieve relevant scientific literature for effective consumption by researchers in a wide range of fields. We provide an analysis of several multi-label document classification models on the LitCovid dataset, a growing collection of 23,000 research papers regarding the novel 2019 coronavirus. We find that pre-trained lan- guage models fine-tuned on this dataset outper- form all other baselines and that BioBERT sur- passes the others by a small margin with micro- F1 and accuracy scores of around 86% and 75% respectively on the test set. We evaluate the data efficiency and generalizability of these models as essential features of any system pre- pared to deal with an urgent situation like the current health crisis. We perform a data ab- lation study to determine how important arti- cle titles are for achieving reasonable perfor- mance on this dataset. Finally, we explore 50 errors made by the best performing models on LitCovid documents and find that they often (1) correlate certain labels too closely together and (2) fail to focus on discriminative sections of the articles; both of which are important is- sues to address in future work. Both data and code are available on GitHub 1. 1 Introduction The COVID-19 pandemic has made it a global pri- ority for research on the subject to be developed at unprecedented rates. Researchers in a wide va- riety of fields, from clinicians to epidemiologists to policymakers, must all have effective access to the most up to date publications in their respective areas. Automated document classification can play an essential role in organizing the stream of articles by fields and topics to facilitate the search process and speed up research efforts. 1https://github.com/dki-lab/ covid19-classification To explore how document classification models can help organize COVID-19 research papers, we use the LitCovid dataset (Chen et al., 2020), a col- lection of 23,000 newly released scientific papers compiled by the NIH to facilitate access to the lit- erature on all aspects of the virus. This dataset is updated daily and every new article is manu- ally assigned one or more of the following 8 cat- egories: General, Transmission Dynamics (Trans- mission), Treatment, Case Report, Epidemic Fore- casting (Forecasting), Prevention, Mechanism and Diagnosis. We leverage these annotations and the articles made available by LitCovid to compile a timely new dataset for multi-label document classi- fication. Apart from addressing the pressing needs of the pandemic, this dataset also offers an interesting document classification dataset which spans differ- ent biomedical specialties while sharing one over- arching topic. This setting is distinct from other biomedical document classification datasets, which tend to exclusively distinguish between biomedical topics such as hallmarks of cancer (Baker et …

Content truncated to 500 words