Analyzing analytical methods: The case of phonology in neural models of spoken language

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4146–4156 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 4146 Analyzing analytical methods: The case of phonology in neural models of spoken language Grzegorz Chrupała Cognitive Science and AI Tilburg University g.chrupala@uvt.nl Bertrand Higy Cognitive Science and AI Tilburg University b.j.r.higy@uvt.nl Afra Alishahi Cognitive Science and AI Tilburg University a.alishahi@uvt.nl Abstract Given the fast development of analysis tech- niques for NLP and speech processing sys- tems, few systematic studies have been con- ducted to compare the strengths and weak- nesses of each method. As a step in this di- rection we study the case of representations of phonology in neural network models of spo- ken language. We use two commonly ap- plied analytical techniques, diagnostic classi- fiers and representational similarity analysis, to quantify to what extent neural activation patterns encode phonemes and phoneme se- quences. We manipulate two factors that can affect the outcome of analysis. First, we in- vestigate the role of learning by comparing neural activations extracted from trained ver- sus randomly-initialized models. Second, we examine the temporal scope of the activations by probing both local activations correspond- ing to a few milliseconds of the speech signal, and global activations pooled over the whole utterance. We conclude that reporting anal- ysis results with randomly initialized models is crucial, and that global-scope methods tend to yield more consistent results and we recom- mend their use as a complement to local-scope diagnostic methods. 1 Introduction As end-to-end architectures based on neural net- works became the tool of choice for processing speech and language, there has been increased in- terest in techniques for analyzing and interpreting the representations emerging in these models. A large array of analytical techniques have been pro- posed and applied to diverse tasks and architectures (Belinkov and Glass, 2019; Alishahi et al., 2019). Given the fast development of analysis tech- niques for NLP and speech processing systems, relatively few systematic studies have been con- ducted to compare the strengths and weaknesses of each methodology and to assess the reliability and explanatory power of their outcomes in controlled settings. This paper reports a step in this direction: as a case study, we examine the representation of phonology in neural network models of spoken language. We choose three different models that process speech signal as input, and analyze their learned neural representations. We use two commonly applied analytical tech- niques: (i) diagnostic models and (ii) representa- tional similarity analysis to quantify to what extent neural activation patterns encode phonemes and phoneme sequences. In our experiments, we manipulate two impor- tant factors that can affect the outcome of analy- sis. One pitfall not always successfully avoided in work on neural representation analysis is the role of learning. Previous work has shown that sometimes non-trivial representations can be found in the activation patterns of randomly initialized, untrained neural networks (Zhang and Bowman, 2018; Chrupała and Alishahi, 2019). Here we in- vestigate the representations of phonology in …

Content truncated to 500 words