Extracting token-level signals of syntactic processing from fMRI - with an application to PoS induction

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pages 747–755, Berlin, Germany, August 7-12, 2016. c⃝2016 Association for Computational Linguistics Extracting token-level signals of syntactic processing from fMRI - with an application to PoS induction Joachim Bingel Maria Barrett Anders Søgaard Centre for Language Technology, University of Copenhagen Njalsgade 140, 2300 Copenhagen S, Denmark {bingel, barrett, soegaard}@hum.ku.dk Abstract Neuro-imaging studies on reading differ- ent parts of speech (PoS) report somewhat mixed results, yet some of them indicate different activations with different PoS. This paper addresses the difficulty of using fMRI to discriminate between linguistic tokens in reading of running text because of low temporal resolution. We show that once we solve this problem, fMRI data contains a signal of PoS distinctions to the extent that it improves PoS induction with error reductions of more than 4%. 1 Introduction A few recent studies have tried to extract mor- phosyntactic signals from measurements of human sentence processing and used this information to improve NLP models. Klerke et al. (2016), for ex- ample, used eye-tracking recordings to regularize a sentence compression model. More related to this work, Barrett et al. (2016) recently used eye- tracking recordings to induce PoS models. How- ever, a weakness of eye-tracking data is that while eye movement surely does reflect the temporal as- pect of cognitive processing, it is only a proxy of the latter and does not directly represent which processes take place in the brain. A recent neuro-imaging study suggests that concrete nouns and verbs elicit different brain sig- natures in the frontocentral cortex, and that con- crete and abstract nouns elicit different brain acti- vation patterns (Moseley and Pulverm¨uller, 2014). Also, for example, concrete verbs activate motor and premotor cortex more strongly than concrete nouns, and concrete nouns activate inferior frontal areas more strongly than concrete verbs. A decade earlier, Tyler et al. (2004) showed that the left in- ferior frontal gyrus was more strongly activated in processing regularly inflected verbs compared to regularly inflected nouns. Such studies suggest that different parts of our brains are activated when reading different parts of speech (PoS). This would in turn mean that neuro-images of readers carry information about the grammatical structure of what they read. In other words, neuro-imaging provides a partial, noisy annotation of the data with respect to mor- phosyntactic category. Say neuro-imaging data of readers was readily available. Would it be of any use to, for exam- ple, engineers interested in PoS taggers for low- resource languages? This is far from obvious. In fact, it is well-known that neuro-imaging data from reading is noisy, in part because the reading signal is not always very distinguishable (Taga- mets et al., 2000), and also because the content of what we read may elicit certain activation in brain regions e.g. related to sensory processing (Boulenger et al., 2006; Gonz´alez et al., 2006). Other researchers such as Borowsky et al. (2013) have also questioned that there are differ- ences, claiming to show that the …

Content truncated to 500 words