Annotation of Emotion Carriers in Personal Narratives

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 138–146 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 138 ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation Nora Hollenstein1, Marius Troendle2, Ce Zhang1, Nicolas Langer2 1 Department of Computer Science, ETH Zurich 2 Department of Psychology, University of Zurich {noraho,ce.zhang}@inf.ethz.ch {m.troendle,n.langer}@psychologie.uzh.ch Abstract We recorded and preprocessed ZuCo 2.0, a new dataset of simultaneous eye-tracking and electroencephalography during natural reading and during annotation. This corpus contains gaze and brain activity data of 739 English sentences, 349 in a normal reading paradigm and 390 in a task-specific paradigm, in which the 18 participants actively search for a semantic relation type in the given sentences as a linguistic annotation task. This new dataset complements ZuCo 1.0 by providing experiments designed to analyze the differences in cognitive processing between natural reading and annotation. The data is freely available here: https://osf.io/2urht/. Keywords: annotation, cognitive methods, corpus, EEG, eye-tracking, human language processing, naturalistic reading, physio- logical data 1. Introduction How humans process language has become increasingly relevant in natural language processing (NLP) since physio- logical data during language understanding is more accessi- ble and recorded with less effort. In this work, we focus on eye-tracking and electroencephalography (EEG) recordings to capture the reading process. On one hand, eye movement data provides millisecond-accurate records about where hu- mans look when they are reading, and is highly correlated with the cognitive load associated with different stages of text processing. On the other hand, EEG records electri- cal brain activity across the scalp and is a direct measure of physiological processes, including language processing. The combination of both measurement methods enables us to study the language understanding process in a more nat- ural setting, where participants read full sentences at a time, in their own speed. Eye-tracking then permits us to define exact word boundaries in the timeline of a subject reading a sentence, allowing the extraction of brain activity signals for each word. Human cognitive language processing data is immensely useful for NLP: Not only can it be leveraged to improve NLP applications (e.g. Barrett et al. (2016) for part- of-speech tagging, or Klerke et al. (2016) for sentence compression), but also to evaluate state-of-the-art machine learning systems. For example, Hollenstein et al. (2019) evaluate word embeddings, or Schwartz et al. (2019) fine- tune language models with brain-relevant bias. Additionally, the availability of labelled data plays a crucial role in all supervised machine learning applications. Phys- iological data can be used to understand and improve the labelling process (e.g. Tokunaga et al. (2017)), and, for instance, to build cost models for active learning scenarios (Tomanek et al., 2010). Is it possible to replace this ex- pensive manual work with models trained on physiological activity data recorded from humans while reading? That is to say, can we find and extract the relevant aspects of text understanding and annotation directly from the source, i.e. Figure 1: Visualization …

Content truncated to 500 words