SpanPredict: Extraction of Predictive Document Spans with Neural Attention

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5234–5258 June 6–11, 2021. ©2021 Association for Computational Linguistics 5234 SpanPredict: Extraction of Predictive Document Spans with Neural Attention Vivek Subramanian, Matthew Engelhard, Samuel Berchuck, Liqun Chen, Ricardo Henao, and Lawrence Carin Duke University {vivek.subramanian, matthew.engelhard, samuel.berchuck, liqun.chen, ricardo.henao, lcarin}@duke.edu Abstract In many natural language processing applica- tions, identifying predictive text can be as im- portant as the predictions themselves. When predicting medical diagnoses, for example, identifying predictive content in clinical notes not only enhances interpretability, but also al- lows unknown, descriptive (i.e., text-based) risk factors to be identified. We here formal- ize this problem as predictive extraction and address it using a simple mechanism based on linear attention. Our method preserves dif- ferentiability, allowing scalable inference via stochastic gradient descent. Further, the model decomposes predictions into a sum of contri- butions of distinct text spans. Importantly, we require only document labels, not ground-truth spans. Results show that our model identifies semantically-cohesive spans and assigns them scores that agree with human ratings, while preserving classification performance. 1 Introduction Attention-based neural network architectures achieve human-level performance in many docu- ment classification tasks. However, understanding model predictions remains challenging. Common feature attribution methods are often inadequate, because the “features” of a document classification model – individual words or their embeddings – tend to have limited or ambiguous meaning in iso- lation, and must instead be interpreted in context. Rather than examining the importance of individ- ual words and passing the contextualization task to the end-user, we may wish to extract distinct spans of text, such as sentences or paragraphs, and quan- tify the effect of each span on model predictions. However, the appropriate span boundaries depend on the document type, and processing all possible spans individually is computationally prohibitive. In some settings, understanding model predic- tions can be as important as the predictions them- selves. When predicting medical diagnoses from clinical notes, for example, attributing predictions to specific note content assures clinicians that the model is not relying on data artifacts that are not clinically meaningful or generalizable. Moreover, this process may illuminate previously unknown risk factors that are described in clinical notes but not captured in a structured manner. Our work is motivated by the problem of autism spectrum dis- order (ASD) diagnosis, in which many early symp- toms are behavioral rather than physiologic, and are documented in clinical notes using multiple- word descriptions, not individual terms. Morever, extended and nuanced descriptions are important in many common document classification tasks, for instance, the scoring of movie or food reviews. Identifying important spans of text is a recurring theme in natural language processing. In extractive summarization, a document summary is created by selecting and concatenating important spans within a document (Narayan et al., 2018); and in many question answering tasks, including in the Stanford Question Answering Dataset (Rajpurkar et al., 2018), the goal is to identify a span within a paragraph of …

Content truncated to 500 words