MasonNLP+ at SemEval-2023 Task 8: Extracting Medical Questions, Experiences and Claims from Social Media using Knowledge-Augmented Pre-trained Language Models

Proceedings of the The 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 2143–2152 July 13-14, 2023 ©2023 Association for Computational Linguistics MasonNLP+ at SemEval-2023 Task 8: Extracting Medical Questions, Experiences and Claims from Social Media using Knowledge-Augmented Pre-trained Language Models Giridhar Kaushik Ramachandran1∗, Haritha Gangavarapu2∗, Kevin Lybarger1, Özlem Uzuner1 1Department of Information Sciences and Technology, George Mason University 2 CGI Inc. Abstract In online forums like Reddit, users share their experiences with medical conditions and treat- ments, including making claims, asking ques- tions, and discussing the effects of treatments on their health. Building systems to under- stand this information can effectively monitor the spread of misinformation and verify user claims. The Task-8 of the 2023 International Workshop on Semantic Evaluation focused on medical applications, specifically extracting pa- tient experience- and medical condition-related entities from user posts on social media. The Reddit Health Online Talk (RedHot) corpus contains posts from medical condition-related subreddits with annotations characterizing the patient experience and medical conditions. In Subtask-1, patient experience is characterized by personal experience, questions, and claims. In Subtask-2, medical conditions are character- ized by population, intervention, and outcome. For the automatic extraction of patient experi- ences and medical condition information, as a part of the challenge, we proposed language- model-based extraction systems that ranked 3rd on both subtasks’ leaderboards. In this work, we describe our approach and, in addition, ex- plore the automatic extraction of this informa- tion using domain-specific language models and the inclusion of external knowledge. 1 Introduction Social media platforms, like Reddit, allow users to share their experiences pseudonymously. On these platforms, users tend to share different kinds of in- formation, including questions, their personal expe- riences using certain prescribed drugs, or how they deal with the different symptoms caused by their medical condition. Users may also make claims about certain clinical and non-clinical interventions (e.g. home remedies) and the associated outcomes. While this information can provide insight into ∗ Equal contribution medical conditions and how the condition presen- tation among users, their personal experience may be inconsistent with current medical evidence. Thus, effective monitoring of these medical condition-related forums is needed to prevent the spread of misinformation and corroborate various user claims against scientific evidence. Information from social media could also augment current sci- entific understanding of conditions and treatments. Natural language processing (NLP) information extraction (IE) techniques can extract information from user posts, which could be incorporated into systems for monitoring and preventing user mis- conception. The RedHot (Wadhwa et al., 2022) corpus consists of over 22,000 user posts from 24 medical condition-related subreddits annotated for entities related to patient experiences, claims, inter- ventions, and outcomes. The posts are annotated in two stages: (1) Subtask-1 annotations include user claims, personal experiences, questions, and claim-based personal experiences and (2) Subtask- 2 annotations contain more granular information through entities related to patient population, inter- vention, and outcome (PIO). In this work, we explore the automatic extrac- tion of these entities from a subset of the RedHot corpus as released through …

Content truncated to 500 words