EEG responds to conceptual stimuli and corpus semantics

Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 619–627, Singapore, 6-7 August 2009. c⃝2009 ACL and AFNLP EEG responds to conceptual stimuli and corpus semantics Brian Murphy CIMeC, University of Trento Rovereto 38068, Italy brian.murphy@unitn.it Marco Baroni CIMeC, University of Trento Rovereto 38068, Italy marco.baroni@unitn.it Massimo Poesio CIMeC, University of Trento Rovereto 38068, Italy massimo.poesio@unitn.it Abstract Mitchell et al. (2008) demonstrated that corpus-extracted models of semantic knowledge can predict neural activation patterns recorded using fMRI. This could be a very powerful technique for evaluating conceptual models extracted from corpora; however, fMRI is expensive and imposes strong constraints on data collection. Following on experiments that demonstrated that EEG activation patterns encode enough information to discriminate broad conceptual categories, we show that corpus-based semantic rep- resentations can predict EEG activation patterns with significant accuracy, and we evaluate the relative performance of different corpus-models on this task. 1 Introduction Models of semantic relatedness induced from cor- pus data have proven effective in a number of em- pirical tasks (Sahlgren, 2006) and there is increas- ing interest in whether distributional information extracted from corpora correlates with aspects of speakers’ semantic knowledge: see Lund and Burgess (1996), Landauer and Dumais (1997), Al- muhareb (2006), Pad´o and Lapata (2007), Schulte im Walde (2008), among many others. For this purpose, corpus models have been tested on data- sets that are based on semantic judgements (met- alinguistic or meta-cognitive intuitions about syn- onymy, semantic distance, category-membership) or behavioural experiments (semantic priming, property generation, free association). While all these data are valuable, they are indirect reflec- tions of semantic knowledge, and when the pre- dictions they make diverge from those of corpora, interpretation is problematic: is the corpus model missing essential aspects of semantics, or are non- semantic factors biasing the data elicited from in- formants? Reading semantic processes and representations directly from the brain would be an ideal way to get around these limitations. Until recently, anal- ysis of linguistic quantities using neural data col- lected with EEG (measurement at the scalp of volt- ages induced by neuronal firing) or fMRI (mea- surement of changes of oxygen concentrations in the brain tied to cognitive processes) had neither the advantages of corpora (scale) nor of infor- mants (finer grained judgements). However, some clear patterns of differential ac- tivity have been found for broad semantic classes. Viewing images of natural (typically animals and plants) and non-natural (typically artefacts like tools or vehicles) objects elicits different loci of activity in fMRI (Martin and Chao, 2001) and EEG (Kiefer, 2001), that persist across partici- pants. Differences have also been found in re- sponse to auditorily or visually presented words of different lexical classes, such as abstract/concrete, and verb/noun (Pulverm¨uller, 2002). But interpre- tation of such group results remains somewhat dif- ficult, as they may be consistent with more than one distinction: the natural/artefactual division just mentioned, may rather be between living/non- living entities, dynamic/static entities, or be based on embodied experience (e.g. manipulable or not). More recently, …

Content truncated to 500 words