Higher-order Comparisons of Sentence Encoder Representations

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5838–5845, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5838 Higher-order Comparisons of Sentence Encoder Representations Mostafa Abdou† ∗ Artur Kulmizev♣ Felix Hill♦ Daniel M. Low♠ Anders Søgaard† †Department of Computer Science, University of Copenhagen ♣Department of Linguistics and Philology, Uppsala University ♦DeepMind ♠Program in Speech and Hearing Bioscience and Technology, Harvard Medical School-MIT Abstract Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different mea- surement modalities (e.g., fMRI, electrophys- iology, behavior). As a framework, RSA has several advantages over existing approaches to interpretation of language encoders based on probing or diagnostic classification: namely, it does not require large training samples, is not prone to overfitting, and it enables a more transparent comparison between the represen- tational geometries of different models and modalities. We demonstrate the utility of RSA by establishing a previously unknown cor- respondence between widely-employed pre- trained language encoders and human process- ing difficulty via eye-tracking data, showcas- ing its potential in the interpretability toolbox for neural models. 1 Introduction Examining the parallels between human and ma- chine learning is a natural way for us to better un- derstand the former and track our progress in the latter. The “black box” aspect of neural networks has recently inspired a large body of work related to interpretability, i.e. understanding of represen- tations that such models learn. In NLP, this push has been largely motivated by linguistic questions, such as: what linguistic properties are captured by neural networks? and to what extent do decisions made by neural models reflect established linguis- tic theories? Given the relative recency of such questions, much work in the domain so far has been focused on the context of models in isolation (e.g. what does model X learn about linguistic phenomenon Y?) In order to more broadly under- stand models’ representational tendencies, how- ever, it is vital that such questions be formed not only with other models in mind, but also other rep- resentational methods and modalities (e.g. behav- ioral data, fMRI measurements, etc.). In context of the latter concern, the present-day interpretability toolkit has not yet been able to afford a practical way of reconciling this. In this work, we employ Representational Simi- larity Analysis (RSA) as a simple method of inter- preting neural models’ representational spaces as they relate to other models and modalities. In par- ticular, we conduct an experiment wherein we in- vestigate the correspondence between human pro- cessing difficulty (as reflected by gaze fixation measurements) and the representations induced by popular pretrained language models. In our exper- iments, we hypothesize that there exists an over- lap between the sentences which are difficult for humans to process and those for which per-layer encoder representations are least correlated. Our intuition is that such sentences may exhibit factors such as low-frequency vocabulary, lexical ambiguity, and syntactic complexity (e.g. multi- …

Content truncated to 500 words