Enriching Epidemiological Thematic Features For Disease Surveillance Corpora Classification
15199Authors
nan
Abstract
AbstractWe present EpidBioBERT, a biosurveillance epidemiological document tagger for disease surveillance over PADI-Web system. Our model is trained on PADI-Web corpus which contains news articles on Animal Diseases Outbreak extracted from the web. We train a classifier to discriminate between relevant and irrelevant documents based on their epidemiological thematic feature content in preparation for further epidemiology information extraction. Our approach proposes a new way to perform epidemiological document classification by enriching epidemiological thematic features namely disease, host, location and date, which are used as inputs to our epidemiological document classifier. We adopt a pre-trained biomedical language model with a novel fine tuning approach that enriches these epidemiological thematic features. We find these thematic features rich enough to improve epidemiological document classification over a smaller data set than initially used in PADI-Web classifier. This improves the classifiers ability to avoid false positive alerts on disease surveillance systems. To further understand information encoded in EpidBioBERT, we experiment the impact of each epidemiology thematic feature on the classifier under ablation studies. We compare our biomedical pre-trained approach with a general language model based model finding that thematic feature embeddings pre-trained on general English documents are not rich enough for epidemiology classification task. Our model achieves an F1-score of 95.5% over an unseen test set, with an improvement of +5.5 points on F1-Score on the PADI-Web classifier with nearly half the training data set.
Keywords
Full Content
Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 22–28 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 22 Priming Ancient Korean Neural Machine Translation Chanjun Park1,2∗, Seolhwa Lee3∗, Jaehyung Seo1, Hyeonseok Moon1, Sugyeong Eo1, Heuiseok Lim1† 1Korea University 2Upstage 3University of Copenhagen {bcj1210, seojae777, glee889, djtnrud, limhseok}@korea.ac.kr chanjun.park@upstage.ai sele@di.ku.dk Abstract In recent years, there has been an increasing need for the restoration and translation of historical languages. In this study, we attempt to translate historical records in ancient Korean language based on neural machine translation (NMT). Inspired by priming, a cognitive science theory that two different stimuli influence each other, we propose novel priming ancient-Korean NMT (AKNMT) using bilingual subword embedding initialization with structural property awareness in the ancient documents. Finally, we obtain state-of-the-art results in the AKNMT task. To the best of our knowledge, we confirm the possibility of developing a human-centric model that incorporates the concepts of cognitive science and analyzes the result from the perspective of interference and cognitive dissonance theory for the first time. Keywords: Ancient-Korean Neural Machine Translation, Priming, Neural Machine Translation 1. Introduction Any language changes in appearance over time by gain- ing new meanings of words or through the disappear- ance of old words. This characteristic is called the his- tory of the language. There are words that have different meanings in the past and present. For example, in the Korean language, the original definition of the word “어리다” was “foolish” but has changed in the present day to “little old.” Several studies have been conducted on historical languages that reflect this characteristic (Clanuwat et al., 2019; Assael et al., 2019; Park et al., 2020). Artificial intelligence (AI) research is being conducted based on a cognitive scientific perspective that imitates the structure of the human brain or the brain func- tion analysis, such as reinforcement learning (Kael- bling et al., 1996), neuroplasticity (Lee et al., 2021), cross-language speech perception (Sirois, 2004; Kuhl, 2000), and priming (Pham et al., 2020). With this trend, research is being conducted to incorporate the cogni- tive science theory into the neural machine translation (NMT) (Pham et al., 2020). Based on the examples in previous studies, we reinter- pret the ancient-Korean NMT (AKNMT) with priming (Tulving et al., 1982), a representative cognitive sci- ence theory. Priming is a process in which exposure to a single stimulus affects the response to subsequent stimuli without conscious guidance or intention. For example, when a person thinks of a category of items, similar items are stimulated by the brain. In this study, we propose priming AKNMT influenced by the fact that ∗These authors contributed equally. †This author is the corresponding author. two stimuli work effectively when they are in the same pattern. The method we present here is more human- centric than previous methodologies because it reflects information processing in the human brain in the NMT task. To enable learning of the ancient language and Korean language in the same pattern during NMT …
Content truncated to 500 words
Labeled Text
AbstractWe present EpidBioBERT, a biosurveillance epidemiological document tagger for disease surveillance over PADI-Web system. Our model is trained on PADI-Web corpus which contains news articles on Animal Diseases Outbreak extracted from the web. We train a classifier to discriminate between relevant and irrelevant documents based on their epidemiological thematic feature content in preparation for further epidemiology information extraction. Our approach proposes a new way to perform epidemiological document classification by enriching epidemiological thematic features namely disease, host, location and date, which are used as inputs to our epidemiological document classifier. We adopt a pre-trained biomedical language model with a novel fine tuning approach that enriches these epidemiological thematic features. We find these thematic features rich enough to improve epidemiological document classification over a smaller data set than initially used in PADI-Web classifier. This improves the classifiers ability to avoid false positive alerts on disease surveillance systems. To further understand information encoded in EpidBioBERT, we experiment the impact of each epidemiology thematic feature on the classifier under ablation studies. We compare our biomedical pre-trained approach with a general language model based model finding that thematic feature embeddings pre-trained on general English documents are not rich enough for epidemiology classification task. Our model achieves an F1-score of 95.5% over an unseen test set, with an improvement of +5.5 points on F1-Score on the PADI-Web classifier with nearly half the training data set.
Publication Details
- Paper ID:
- 15199
- Year:
- 2022
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan