MedFilter: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7781–7797, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 7781 MEDFILTER: Improving Extraction of Task-relevant Utterances through Integration of Discourse Structure and Ontological Knowledge Sopan Khosla Shikhar Vashishth Jill Fain Lehman Carolyn Rose Language Technologies Institute Carnegie Mellon University, USA {sopank,svashish,jfl,cprose}@cs.cmu.edu Abstract Information extraction from conversational data is particularly challenging because the task-centric nature of conversation allows for effective communication of implicit informa- tion by humans, but is challenging for ma- chines. The challenges may differ between ut- terances depending on the role of the speaker within the conversation, especially when rel- evant expertise is distributed asymmetrically across roles. Further, the challenges may also increase over the conversation as more shared context is built up through information com- municated implicitly earlier in the dialogue. In this paper, we propose the novel modeling approach MEDFILTER, which addresses these insights in order to increase performance at identifying and categorizing task-relevant ut- terances, and in so doing, positively impacts performance at a downstream information ex- traction task. We evaluate this approach on a corpus of nearly 7,000 doctor-patient con- versations where MEDFILTER is used to iden- tify medically relevant contributions to the dis- cussion (achieving a 10% improvement over SOTA baselines in terms of area under the PR curve). Identifying task-relevant utter- ances benefits downstream medical process- ing, achieving improvements of 15%, 105%, and 23% respectively for the extraction of symptoms, medications, and complaints. 1 Introduction In this paper, we propose a novel modeling ap- proach that embodies insights regarding the organi- zation of task-oriented conversations in order to im- prove performance at utterance classification over SOTA baseline approaches. Task-oriented conver- sations involve sharing task-relevant information that may be useful as the task ensues (Liu et al., 2019a; Kazi and Kahanda, 2019). Unfortunately, human-to-human conversations are less well struc- tured than expository text, which is more often the source material for information extraction and summarization. Expository text is typically struc- tured top-down and organized around information flow. Task-oriented conversations, on the other hand, are typically organized around the task and knowledge of task structure provides an implicit scaffold for understanding. Thus speakers feel free to elide or imply important information rather than making it explicit. The challenges have been well documented (Waitzkin, 1989; Lacson et al., 2006). Prior work in utterance classification is a source of SOTA modeling approaches that perform relatively well despite these challenges while leaving much room for improvement. Our evaluation in this paper specifically focuses on doctor-patient interactions. Doctor-patient inter- actions are task-oriented, expert-layperson interac- tions in which the concerns voiced by the layper- son (e.g., symptoms), the underlying issue iden- tified by the expert (e.g., complaint) and the pre- scribed solutions (e.g., medications) play a crucial part. Customer-service chats are another example of such dialogue. As in the general case, topic switching abounds: the doctor may jump from a question about a symptom to a statement providing initial assessment then back again, with or without …

Content truncated to 500 words