Effective Convolutional Attention Network for Multi-label Clinical Document Classification

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5941–5953 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 5941 Effective Convolutional Attention Network for Multi-label Clinical Document Classification Yang Liu1, Hua Cheng1, Russell Klopfer1, Thomas Schaaf1, Matthew R. Gormley2 13M Health Information Systems, 2Carnegie Mellon University tomdzh1@gmail.com, {hcheng, rklopfer, tschaaf}@mmm.com, mgormley@cs.cmu.edu Abstract Multi-label document classification (MLDC) problems can be challenging, especially for long documents with a large label set and a long-tail distribution over labels. In this paper, we present an effective convolutional attention network for the MLDC problem with a focus on medical code prediction from clinical doc- uments. Our innovations are three-fold: (1) we utilize a deep convolution-based encoder with the squeeze-and-excitation networks and residual networks to aggregate the informa- tion across the document and learn meaning- ful document representations that cover dif- ferent ranges of texts; (2) we explore multi- layer and sum-pooling attention to extract the most informative features from these multi- scale representations; (3) we combine binary cross entropy loss and focal loss to improve performance for rare labels. We focus our evaluation study on MIMIC-III, a widely used dataset in the medical domain. Our models outperform prior work on medical coding and achieve new state-of-the-art results on multiple metrics. We also demonstrate the language in- dependent nature of our approach by applying it to two non-English datasets. Our model out- performs prior best model and a multilingual Transformer model by a substantial margin. 1 Introduction In multi-label document classification (MLDC), we have a set of labeled data {X, Y}, X ∈ RN×D and Y ∈ RN×L, where N is the number of documents, D is the feature dimension size of each document and L is the total number of labels. The ith row of Y is a multi-hot vector representing the set of labels associated with the ith document. The task is to learn a mapping between X and Y so that the labels of each document are predicted correctly. MLDC has a great number of practical appli- cations, one of which is automatic medical cod- ing, where a patient encounter containing multi- ple records are assigned with appropriate medical codes. A large number of medical encounters need to be coded for billing purposes everyday. Profes- sional clinical coders often use rule-based or simple ML-based systems to assign billing codes, but the large code space (viz. the ICD-10 code system with over 90,000 codes) and long documents are challenging for ML models. In addition, coding re- quires extracting useful information from specific locations across the entire encounter to support the assigned codes. Consequently, effective mod- els with the capability of handling these challenges will have an immense impact in the medical domain by helping to reduce coding cost, improve coding accuracy and increase customer satisfaction. Deep learning methods have been demonstrated to produce the state-of-the-art outcomes on bench- mark MLDC and medical coding tasks (You et al., 2019a; Mullenbach et al., 2018; Chang et al., 2020), but demands remain for …

Content truncated to 500 words