Interpreting Word-Level Hidden State Behaviour of Character-Level LSTM Language Models

Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 258–266 Brussels, Belgium, November 1, 2018. c⃝2018 Association for Computational Linguistics 258 Interpreting Word-Level Hidden State Behaviour of Character-Level LSTM Language Models Avery Hiebert†∗, Cole Peterson†, Alona Fyshe‡, Nishant A. Mehta† †Department of Computer Science, University of Victoria, Canada ‡Computing Science / Psychology Departments, University of Alberta, Canada averyhiebert@gmail.com, cpeterso@uvic.ca alona@ualberta.ca, nmehta@uvic.ca Abstract While Long Short-Term Memory networks (LSTMs) and other forms of recurrent neural network have been successfully applied to lan- guage modeling on a character level, the hid- den state dynamics of these models can be difficult to interpret. We investigate the hid- den states of such a model by using the HDB- SCAN clustering algorithm to identify points in the text at which the hidden state is similar. Focusing on whitespace characters prior to the beginning of a word reveals interpretable clus- ters that offer insight into how the LSTM may combine contextual and character-level infor- mation to identify parts of speech. We also introduce a method for deriving word vectors from the hidden state representation in order to investigate the word-level knowledge of the model. These word vectors encode meaning- ful semantic information even for words that appear only once in the training text. 1 Introduction Recurrent Neural Networks (RNNs), including Long Short-Term Memory (LSTM) networks (Hochreiter and Schmidhuber, 1997; Gers et al., 2000), have been widely applied to natural lan- guage processing tasks including character-level language modeling (Mikolov et al., 2012; Graves, 2013). However, like other types of neural net- works, the hidden states and behaviour of a given LSTM can be difficult to understand and interpret, due to both the distributed nature of the hidden state representations and the relatively opaque re- lationship between the hidden state and the final output of the network. It is also not clear how a character-level LSTM language model takes ad- vantage of orthographic patterns to infer higher- level information. ∗Corresponding author In this paper, we investigate the hidden state dy- namics of a character-level LSTM language model both directly and — through the use of output gate activations — indirectly. As an overview, our main contributions are: 1. We use clustering to investigate similar hid- den states (and output gate activations) at dif- ferent points in a text, paying special atten- tion to whitespace characters. We provide in- sight into the model’s awareness of both or- thographic patterns and word-level grammat- ical information. 2. Inspired by our findings from clustering, we introduce a method for extracting meaning- ful word embeddings from a character-level model, allowing us to investigate the word- level knowledge of the model. First, we use the HDBSCAN clustering algo- rithm (Campello et al., 2013) to reveal locations within a text at which the hidden state of the LSTM is similar, or at which a similar combina- tion of cell state dimensions is relevant (as deter- mined by output gates). Interestingly, focusing on moments when the network must predict the …

Content truncated to 500 words