Extracting Chemical-Protein Interactions via Calibrated Deep Neural Network and Self-training

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2086–2095 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 2086 Extracting Chemical–Protein Interactions via Calibrated Deep Neural Network and Self-training Dongha Choi and Hyunju Lee Data Mining and Computational Biology Lab. School of Electrical Engineering and Comuter Science and Artificial Intelligence Graduate School Gwangju Institute of Science and Technology, Gwangju, Republic of Korea {dongha528,hyunjulee}@gist.ac.kr Abstract The extraction of interactions between chem- icals and proteins from several biomedical ar- ticles is important in many fields of biomedi- cal research such as drug development and pre- diction of drug side effects. Several natural language processing methods, including deep neural network (DNN) models, have been ap- plied to address this problem. However, these methods were trained with hard-labeled data, which tend to become over-confident, lead- ing to degradation of the model reliability. To estimate the data uncertainty and improve the reliability, “calibration” techniques have been applied to deep learning models. In this study, to extract chemical–protein interactions, we propose a DNN-based approach incorpo- rating uncertainty information and calibration techniques. Our model first encodes the in- put sequence using a pre-trained language- understanding model, following which it is trained using two calibration methods: mixup training and addition of a confidence penalty loss. Finally, the model is re-trained with augmented data that are extracted using the estimated uncertainties. Our approach has achieved state-of-the-art performance with re- gard to the Biocreative VI ChemProt task, while preserving higher calibration abilities than those of previous approaches. Further- more, our approach also presents the possibili- ties of using uncertainty estimation for perfor- mance improvement. 1 Introduction In the biomedical domain, there exist several enti- ties, such as genes, chemicals, and diseases, that are closely related to each other. Therefore, extracting the relationships among these entities is critical for biomedical research, particularly in fields such as construction of a knowledge base or drug develop- ment. Biomedical text data, including PubMed ab- stracts, usually contain information about biomedi- cal entities and their relationships with each other. Thus, various natural language processing models, particularly deep learning models, are applied to biomedical text data to extract the relationships among these entities, as a kind of classification task. ChemProt corpus (Krallinger et al., 2017) is the first corpus dataset for chemical–protein (gene) re- lationship extraction, which has been conducted by BioCreative VI organizers. These organizers annotated all entity offsets of chemical and protein mentions and relationship types between chemicals and proteins (Chemical-Protein Relations, CPR). There exist 10 groups of the relationship types, and five of these (CPR:3, CPR:4, CPR:5, CPR:6, and CPR:9) were used in the evaluation. All models for extracting relationships from ChemProt data are designed as classifiers. In a deep learning-based multi-class classifier, the out- put probability distribution for each class is calcu- lated through the Softmax function. In the training step, the model is trained to maximize the output probability of the correct class. However, some studies reported that the deep learning classifier trained with hard-labeled data (1 for …

Content truncated to 500 words