LingX at ROCLING 2023 MultiNER-Health Task: Intelligent Capture of Chinese Medical Named Entities by LLMs

LingX at ROCLING 2023 MultiNER-Health Task: Intelligent Capture of Chinese Medical Named Entities by LLMs Xuelin Wang College of Chinese Language and Culture Jinan University Guangzhou, China wangxuelin@stu2022.jnu.edu.cn Qihao Yang� School of Computer Science South China Normal University Guangzhou, China charlesyeung@m.scnu.edu.cn Abstract Medical Named Entity Recognition (NER) stands as a pivotal technique within the realm of medicine, cate sequence labeling.encompassing intri- Profound medi- cal knowledge acumen and accurate de- marcation of entity boundaries constitute the principal challenges of this task. In contrast to the English context, Chinese medical NER poses even greater chal- lenges. Presently, prominent Large Lan- guage Models (LLMs) such as ChatGPT have ushered in prospects for various down- stream tasks in natural language process- ing. This paper introduces a novel re- search approach to explore the poten- tial and performance of LLMs in cap- turing named entities: the transforma- tion of sequence labeling into entity ex- traction. In this study, typical medi- cal NER datasets in the BIO format are adapted into prompts suitable for LLMs, and through instruct-tuning, two fine- tuned LLMs for medical entity extraction are constructed. Experimental findings un- veiled that our approach attains an aver- age F1 score of 57.02% in ROCLING-2023 MultiNER-Health Task, outperforming the zero-shot performance of ChatGPT-3.5 (39.32%). Furthermore, comparative ex- perimentation substantiates the robust generalization capability of the proposed approach. 1 Introduction Medical Named Entity Recognition (NER), a fundamental information extraction task in the field of medical natural language process- ing, aims to extract predefined entities such as “instrument,” “drug,” and ”diseases” from sentences (Liu et al., 2022). Medical NER is �Corresponding author. conventionally framed as a sequence labeling problem, wherein the BIO (Begin, Inside, Out- side) scheme is commonly employed to jointly predict entity boundaries and category labels within sentences (Lee and Lu, 2021). Owing to the intricacy of medical texts, the research landscape of medical NER continues to grap- ple with substantial challenges (Ji et al., 2020), such as the absence of standardized nomen- clature for medical entities and the continu- ous emergence of novel medical entities (Ji et al., 2019). Hence, medical NER models typically necessitate specialized and continu- ously refined medical expertise, as well as pre- cise entity boundary recognition capabilities- both of which stand as primary challenges of this task. Furthermore, Chinese text lacks in- herent delimiters. In comparison to English text, Chinese text is more prone to instances of incomplete semantic information or even am- biguity due to inaccuracies in word segmen- tation (Wang et al., 2020). To advance re- search in Chinese medical NER, ROCLING- 2022 Shared Task (Lee et al., 2022a) and ROCLING-2023 MultiNER-Health Task(Lee et al., 2023) have established a competitive platform for Chinese medical NER encompass- ing 10 entity categories. This platform in- cludes curated training sets, standardized test- ing sets, and evaluation metric suites. Recently, prominent Large Language Mod- els (LLMs), exemplified by ChatGPT (Ouyang et al., 2022), have showcased impressive ca- pabilities in natural language comprehension and generation, both within the academic and industrial domains (Wang et …

Content truncated to 500 words