Adapting Event Extractors to Medical Data: Bridging the Covariate Shift

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 2963–2975 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 2963 Adapting Event Extractors to Medical Data: Bridging the Covariate Shift Aakanksha Naik1 Jill Lehman2 1Language Technologies Institute, Carnegie Mellon University 2Human-Computer Interaction Institute, Carnegie Mellon University {anaik,jfl,cprose}@andrew.cmu.edu Carolyn Ros´e1 Abstract We tackle the task of adapting event extrac- tors to new domains without labeled data, by aligning the marginal distributions of source and target domains. As a testbed, we create two new event extraction datasets using En- glish texts from two medical domains: (i) clini- cal notes, and (ii) doctor-patient conversations. We test the efficacy of three marginal align- ment techniques: (i) adversarial domain adap- tation (ADA), (ii) domain adaptive fine-tuning (DAFT), and (iii) a new instance weighting technique based on language model likelihood scores (LIW). LIW and DAFT improve over a no-transfer BERT baseline on both domains, but ADA only improves on notes. Deeper anal- ysis of performance under different types of shifts (e.g., lexical shift, semantic shift) ex- plains some of the variations among models. Our best-performing models reach F1 scores of 70.0 and 72.9 on notes and conversations respectively, using no labeled target data. 1 Introduction Events are an important phenomenon in the field of computational semantics. They offer an intuitive mechanism for constructing structured representa- tions of text, which can be used for downstream tasks such as question answering and summariza- tion. Events also embody a crucial function of lan- guage: the ability to report happenings. Narratives from many diverse domains (e.g., news articles, literary texts, clinical notes) use events as basic building blocks. These characteristics make event extraction a key sub-task of interest for text under- standing pipelines in multiple domains. Despite its importance, building high-performing and gener- alizable systems for event extraction has remained an elusive goal. One of the major hurdles is that the notion of what counts as an important event is usually task-specific or domain-specific (some- times both). For example, to build a system that can track a patient’s disease progression from clin- ical notes, event extractors only need to focus on extracting medical events relevant to that illness. This task/domain specificity has encouraged prior work to focus on specific event types (Grishman and Sundheim, 1996; Doddington et al., 2004; Kim et al., 2008) or domains (Pustejovsky et al., 2003b; Sims et al., 2019). Owing to this narrow focus, su- pervised event extractors often fail to adapt to new domains or event types (Keith et al., 2017). Un- supervised event extractors that use syntactic rule- based modules (Saur´ı et al., 2005; Chambers et al., 2014), conversely, have a tendency to over-generate by labeling most verbs and nouns as events. In this work, we try to achieve a balance be- tween these extremes by adapting event extractors using unsupervised domain adaptation techniques. We also study the behavior of these techniques under various types of linguistic shifts (e.g., lexi- cal shift, semantic shift) to gain …

Content truncated to 500 words