Medical Knowledge-enriched Textual Entailment Framework

Proceedings of the 28th International Conference on Computational Linguistics, pages 1795–1801 Barcelona, Spain (Online), December 8-13, 2020 1795 Medical Knowledge-enriched Textual Entailment Framework Shweta Yadav∗, Vishal Pallagani‡, Amit Sheth‡ ∗LHNCBC, U.S. National Library of Medicine, MD, USA ‡ University of South Carolina, SC, USA ∗shweta.shweta@nih.gov, ‡{VISHAL,AMIT}@sc.edu Abstract One of the cardinal tasks in achieving robust medical question answering systems is textual en- tailment. The existing approaches make use of an ensemble of pre-trained language models or data augmentation, often to clock higher numbers on the validation metrics. However, two major shortcomings impede higher success in identifying entailment: (1) understanding the focus/intent of the question and (2) ability to utilize the real-world background knowledge to capture the con- text beyond the sentence. In this paper, we present a novel Medical Knowledge-Enriched Textual Entailment framework that allows the model to acquire a semantic and global representation of the input medical text with the help of a relevant domain-specific knowledge graph. We evaluate our framework on the benchmark MEDIQA-RQE dataset and manifest that the use of knowledge- enriched dual-encoding mechanism help in achieving an absolute improvement of 8.27% over SOTA language models. We have made the source code available here.1 1 Introduction The entailment task is similar with natural language inference (NLI), involves identifying the semantic similarity between two natural language texts, premise (P) and hypothesis (H). The NLI task’s effective- ness is crucial for developing a robust natural language understanding system that functions at a human level (Ben Abacha et al., 2019; Abacha and Demner-Fushman, 2016; Romanov and Shivade, 2018). Recent literature suggests the use of contemporary language models (LMs) (Devlin et al., 2018; Beltagy et al., 2019), often ensembled, to achieve better performance (Zhu et al., 2019; Bhaskar et al., 2019; Xu et al., 2019) on the NLI task. However, our qualitative interpretation of the dataset and results suggests that LMs fails when it comes to textual entailment (TE), despite being the on-demand language model. The limitations belong to two major categories: • Multiple word form: Medical text offers high degree of variability in the form of synonym and abbreviated words. The same can be witnessed in Table-1, where BERT (Devlin et al., 2018) is unable to predict the entailment between P and H as they have different terms - ‘Kartagener’s Syndrome’ and ‘Primary Ciliary Dyskinesia’, while both are synonym. • Focus/Intent understanding: Given P and H, LMs often fails to capture the focus/intent of both the sentences. Table-1 shows an example, where the focus of P and H are misunderstood. It can be seen that P emphasizes the possibility of ‘atypical pneumonia’ occurring within a month after treatment, whereas H talks about the possible treatments for the disease. These findings indicate that existing LMs lack semantic interpretation of the input, which is crucial in the inferencing tasks. In this paper, we deal with the question: Does the medical textual entailment task benefit from the external domain knowledge to distinguish semantically identical medical sentences to recognize entailment? To address this question …

Content truncated to 500 words