Word-Level Loss Extensions for Neural Temporal Relation Classification

Proceedings of the 27th International Conference on Computational Linguistics, pages 3436–3447 Santa Fe, New Mexico, USA, August 20-26, 2018. 3436 Word-Level Loss Extensions for Neural Temporal Relation Classification Artuur Leeuwenberg and Marie-Francine Moens Department of Computer Science KU Leuven, Belgium {tuur.leeuwenberg, sien.moens}@cs.kuleuven.be Abstract Unsupervised pre-trained word embeddings are used effectively for many tasks in natural lan- guage processing to leverage unlabeled textual data. Often these embeddings are either used as initializations or as fixed word representations for task-specific classification models. In this work, we extend our classification model’s task loss with an unsupervised auxiliary loss on the word-embedding level of the model. This is to ensure that the learned word representations con- tain both task-specific features, learned from the supervised loss component, and more general features learned from the unsupervised loss component. We evaluate our approach on the task of temporal relation extraction, in particular, narrative containment relation extraction from clin- ical records, and show that continued training of the embeddings on the unsupervised objective together with the task objective gives better task-specific embeddings, and results in an improve- ment over the state of the art on the THYME dataset, using only a general-domain part-of-speech tagger as linguistic resource. 1 Introduction Word representations in the form of continuous vectors are often pre-trained on large amounts of raw text to learn general word features, using unsupervised objectives. These representations are then used in supervised models for various classification tasks. However, such tasks sometimes require very spe- cific features that may not have been captured by the unsupervised objective. In other domains such as computer vision, representations are often learned jointly from multiple resources for classification. In this work, we explore the possibility to exploit learning signals from both settings to construct better task-oriented word representations, and obtain a better relation classification model. The main task in this work is the extraction of narrative containment relations (CR) from English clinical texts, as annotation of clinical data is costly and it is therefor crucial to fully exploit both the labeled as well as the unlabeled data that is available. The aim of CR extraction is to find if, given events A and B, event A is temporally contained in event B (i.e. if event A happens within the time span of event B). An example of such relation is given in Fig. 1, where the model should predict all containment edges given the entities (events and temporal expressions), by classifying each pair of entities as containment or no containment. Temporal relation classification in clinical text is a very important task in the secondary use of clinical data from electronic health records. The patient time-line is crucial for making a good patient prognosis and clinical decision support (Onisko et al., 2015). This task has already been addressed Figure 1: A sentence annotated with events, temporal expressions (Timex3), and containment relations. This work is licensed under a Creative Commons Attribution 4.0 International License. License details: http:// creativecommons.org/licenses/by/4.0/ 3437 in three iterations of the Clinical TempEval …

Content truncated to 500 words