Sequential Span Classification with Neural Semi-Markov CRFs for Biomedical Abstracts

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 871–877 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 871 Sequential Span Classification with Neural Semi-Markov CRFs for Biomedical Abstracts Kosuke Yamada1 Tsutomu Hirao2 Ryohei Sasano1 Koichi Takeda1 Masaaki Nagata2 1Graduate School of Informatics, Nagoya University, Japan 2NTT Communication Science Laboratories, NTT Corporation, Japan yamada.kosuke@c.mbox.nagoya-u.ac.jp, {tsutomu.hirao.kp,masaaki.nagata.et}@hco.ntt.co.jp, {sasano,takedasu}@i.nagoya-u.ac.jp Abstract Dividing biomedical abstracts into several seg- ments with rhetorical roles is essential for sup- porting researchers’ information access in the biomedical domain. Conventional methods have regarded the task as a sequence label- ing task based on sequential sentence classi- fication, i.e., they assign a rhetorical label to each sentence by considering the context in the abstract. However, these methods have a critical problem: they are prone to misla- bel longer continuous sentences with the same rhetorical label. To tackle the problem, we propose sequential span classification that as- signs a rhetorical label, not to a single sen- tence but to a span that consists of continuous sentences. Accordingly, we introduce Neural Semi-Markov Conditional Random Fields to assign the labels to such spans by consider- ing all possible spans of various lengths. Ex- perimental results obtained from PubMed 20k RCT and NICTA-PIBOSO datasets demon- strate that our proposed method achieved the best micro sentence-F1 score as well as the best micro span-F1 score. 1 Introduction Dividing documents into several rhetorical seg- ments is a fundamental task in natural language pro- cessing (NLP). For example, abstracts in PubMed, a database of the biomedical literature, can be di- vided into rhetorical segments such as “Objective”, “Methods”, “Results”, and “Conclusions”. Ab- stracts segmented for each rhetorical role allows us to exploit advanced search. That is, researchers can easily find information by utilizing the structured queries such as “find abstracts that contain ‘Covid- 19’ in ‘Objective’ and ‘Remdesivir’ in ‘Methods’”. Furthermore, the technique can also be used for NLP applications such as academic writing support (Huang and Chen, 2017), scientific trend analysis (Prabhakaran et al., 2016), and question-answering (Guo et al., 2013). Most previous methods in PubMed have re- garded the task as a sequence labeling, namely se- quential sentence classification, that assigns rhetor- ical labels with a B(egin)/I(nside) tag set to each sentence while considering the context in the abstract. To this end, some statistical meth- ods with hand-engineered features have been pro- posed, including Hidden Markov Models (HMMs) (Lin et al., 2006) and Conditional Random Fields (CRFs) (Hirohata et al., 2008; Kim et al., 2011; Hassanzadeh et al., 2014). Recently, with the suc- cess of neural network models for NLP tasks, Der- noncourt et al. (2017) and Jin and Szolovits (2018) have employed BiLSTMs to obtain sentence em- beddings based on word embeddings and CRFs for assigning labels to the sentences. Cohan et al. (2019) employed a pre-trained language model, SCIBERT (Beltagy et al., 2019), which is a variant of BERT (Devlin et al., 2019) trained with scientific papers, to improve the performance of classifica- tion without CRFs. Previous methods cast the segmentation …

Content truncated to 500 words