Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 4295 - 4309 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze? Oliver Eberle∗,1, Stephanie Brandl∗,1,2, Jonas Pilot1, Anders Søgaard2 1Machine Learning Group, TU Berlin, Germany 2University of Copenhagen, Denmark oliver.eberle@tu-berlin.de, {brandl, soegaard}@di.ku.dk ∗Authors contributed equally. Abstract Learned self-attention functions in state-of-the- art NLP models often correlate with human attention. We investigate whether self-attention in large-scale pre-trained language models is as predictive of human eye fixation patterns dur- ing task-reading as classical cognitive models of human attention. We compare attention func- tions across two task-specific reading datasets for sentiment analysis and relation extraction. We find the predictiveness of large-scale pre- trained self-attention for human attention de- pends on ‘what is in the tail’, e.g., the syntactic nature of rare contexts. Further, we observe that task-specific fine-tuning does not increase the correlation with human task-specific read- ing. Through an input reduction experiment we give complementary insights on the spar- sity and fidelity trade-off, showing that lower- entropy attention vectors are more faithful. 1 Introduction The usefulness of learned self-attention functions often correlates with how well it aligns with human attention (Das et al., 2016; Klerke et al., 2016; Bar- rett et al., 2018; Zhang and Zhang, 2019; Klerke and Plank, 2019). In this paper, we evaluate how well attention flow (Abnar and Zuidema, 2020) in large language models, namely BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019) and T5 (Raffel et al., 2020), aligns with human eye fix- ations during task-specific reading, compared to other shallow sequence labeling models (Lecun and Bengio, 1995; Vaswani et al., 2017) and a clas- sic, heuristic model of human reading (Reichle et al., 2003). We compare the learned attention functions and the heuristic model across two task- specific English reading tasks, namely sentiment analysis (SST movie reviews) and relation extrac- tion (Wikipedia), as well as natural reading, us- ing a publicly available dataset with eye-tracking recordings of native speakers of English (Hollen- stein et al., 2018). Contributions We compare human and model attention patterns on both sentiment reading and relation extraction tasks. In our analysis, we com- pare human attention to pre-trained Transformers (BERT, RoBERTa and T5), from-scratch training of two shallow sequence labeling architectures (Le- cun and Bengio, 1995; Vaswani et al., 2017), as well as to a frequency baseline and a heuristic, cog- nitively inspired model of human reading called the E-Z Reader (Reichle et al., 2003). We find that the heuristic model correlates well with human read- ing, as has been reported in Sood et al. (2020b). However when we apply attention flow (Abnar and Zuidema, 2020), the pre-trained Transformer models also reach comparable levels of correlation strength. Further fine-tuning experiments on BERT did not result in increased correlation to human fix- ations. To understand what drives the differences between models, we perform an in-depth analysis of the effect of word predictability and …

Content truncated to 500 words