BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5378–5384 June 6–11, 2021. ©2021 Association for Computational Linguistics 5378 BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification Ishani Mondal Microsoft Research Lavelle Road, Bangalore, India ishani340@gmail.com Abstract Healthcare predictive analytics aids medi- cal decision-making, diagnosis prediction and drug review analysis. Therefore, prediction accuracy is an important criteria which also necessitates robust predictive language mod- els. However, the models using deep learn- ing have been proven vulnerable towards in- significantly perturbed input instances which are less likely to be misclassified by humans. Recent efforts of generating adversaries using rule-based synonyms and BERT-MLMs have been witnessed in general domain, but the ever- increasing biomedical literature poses unique challenges. We propose BBAEG (Biomedi- cal BERT-based Adversarial Example Genera- tion), a black-box attack algorithm for biomed- ical text classification, leveraging the strengths of both domain-specific synonym replacement for biomedical named entities and BERT- MLM predictions, spelling variation and num- ber replacement. Through automatic and hu- man evaluation on two datasets, we demon- strate that BBAEG performs stronger attack with better language fluency, semantic coher- ence as compared to prior work. 1 Introduction Recent studies have exposed the importance of biomedical NLP in the well-being of human-beings, analyzing the critical process of medical decision- making. However, the dialogue managing tools targeted for medical conversations (Zhang et al., 2020), (Campillos Llanos et al., 2017), (Kazi and Kahanda, 2019) between patients and healthcare providers in assisting diagnosis may generate cer- tain insignificant perturbations (spelling errors, paraphrasing), which when fed to the classifier to determine the type of diagnosis required/detecting adverse drug effects/drug recommendation, might provide unreasonable performance. Insignificant ∗The work started when the author was a student at IIT Kharagpur, India. perturbations might also creep in from the casual language expressed in the tweets (Zilio et al., 2020). Thus, the classifier needs to be robust towards these perturbations. Generating adversarial examples in text is chal- lenging compared to computer vision tasks because of (i) discrete nature of input space and (ii) preser- vation of semantic coherence with original text. Initial works for attacking text models relied on introducing errors at the character level or manip- ulating words (Feng et al., 2018) to generate ad- versarial examples. But due to grammatical disflu- ency, these seem very unnatural. Some rule-based synonym replacement strategies (Alzantot et al., 2018), (Ren et al., 2019) have lead to more natu- ral looking examples. (Jin et al., 2019) proposed TextFooler, as a baseline to generate adversaries for text classification models. But, the adversar- ial examples created by TextFooler rely heavily on word-embedding based word similarity replace- ment technique, and not overall sentence seman- tics. Recently, (Garg and Ramakrishnan, 2020) proposed BERT-MLM-based (Devlin et al., 2019) word replacements to create adversaries to better fit the overall context. Despite these advancements, there is much less attention towards making robust predictions in crit- ical domains like biomedical, which comes with its unique challenges. (Araujo …

Content truncated to 500 words