Decoding Part-of-Speech from Human EEG Signals

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 2201 - 2210 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Decoding Part-of-Speech from Human EEG Signals Alex Murphy1, Bernd Bohnet2, Ryan McDonald3, Uta Noppeney4 1University of Birmingham, UK, 2Google Research 3ASAPP, 4Donders Institute for Brain, Cognition and Behaviour murphyalex@gmail.com bohnetbd@google.com rmcdonald@asapp.com uta.noppeney@donders.ru.nl Abstract This work explores techniques to predict Part-of- Speech (PoS) tags from neural signals measured at millisecond resolution with electroencephalogra- phy (EEG) during text reading. We show that infor- mation about word length, frequency and word class is encoded by the brain at different post- stimulus latencies. We then demonstrate that pre- training on averaged EEG data and data augmen- tation techniques boost PoS single-trial EEG de- coding accuracy for Transformers (but not linear SVMs). Applying optimised temporally-resolved decoding techniques we show that Transformers outperform linear SVMs on PoS tagging of uni- gram and bigram data more strongly when infor- mation requires integration across longer time win- dows. 1 Introduction Electro-/Magnetoencephalography (EEG/MEG), which measures neural activity at millisecond res- olution, is a key neuroscientific method to assess how neural representations unfold dynamically in language processing. Early event related potential (ERP) studies that rely on averaging EEG activity across multiple trials have shown that EEG signal magnitude and topography depend on word length, frequency and open vs. closed class. Word length effects arose in EEG at about 150 ms, frequency effects at 200 ms and word class effects from 400- 700 ms (Osterhout et al., 1997; Brown et al., 1999; Neville et al., 1992; Münte et al., 1998; Segalowitz and Lane, 2000; Münte et al., 2001; Dufau et al., 2015). Recent studies were able to predict these and other (e.g. semantic) aspects based on single trial multi-channel EEG/MEG activity (Ling et al., 2019; Chan et al., 2011; King et al., 2020). Impor- tantly, the aim of cognitive neuroscience studies is to dissociate when (i.e. latency) and where (i.e. brain region) specific linguistic information is ex- plicitly encoded in neural activity. Neuroscience studies therefore typically use linear decoders and try to disentangle neural activity for linguistic and non-linguistic dimensions that covary in natural language statistics (e.g. word class vs. length). By contrast, engineering applications mainly aim to maximise performance accuracy, utilising all available information and more powerful non- linear classifiers. Intriguingly, recent studies have shown that adding human eye tracking data (Barrett et al., 2016) or morphosyntactic information ex- tracted from human functional magnetic resonance imaging (fMRI) signals during sentence reading, can substantially improve PoS induction (Bingel et al., 2016). Yet, morphosynactic information ob- tained from fMRI is limited, because fMRI mea- sures only the slow changes in blood oxygenation, peaking 5-6 s after stimulus onset, rather than the rapid neural activity during language processing. Contributions. This interdisciplinary paper de- codes PoS tags from EEG signals with linear SVMs and Transformers, pursuing several aims relevant for neuroscience and/or engineering. Neuroscience-focused Section 3 uses linear SVMs to define the distinct neural representations of …

Content truncated to 500 words