Visualizing and Understanding Neural Models in NLP

Proceedings of NAACL-HLT 2016, pages 681–691, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Visualizing and Understanding Neural Models in NLP Jiwei Li1, Xinlei Chen2, Eduard Hovy2 and Dan Jurafsky1 1Computer Science Department, Stanford University, Stanford, CA 94305, USA 2Language Technology Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA {jiweil,jurafsky}@stanford.edu {xinleic,ehovy}@andrew.cmu.edu Abstract While neural networks have been successfully applied to many NLP tasks the resulting vector- based models are very difficult to interpret. For example it’s not clear how they achieve compositionality, building sentence meaning from the meanings of words and phrases. In this paper we describe strategies for visual- izing compositionality in neural models for NLP, inspired by similar work in computer vision. We first plot unit values to visualize compositionality of negation, intensification, and concessive clauses, allowing us to see well- known markedness asymmetries in negation. We then introduce methods for visualizing a unit’s salience, the amount that it contributes to the final composed meaning from first-order derivatives. Our general-purpose methods may have wide applications for understanding com- positionality and other semantic properties of deep networks. 1 Introduction Neural models match or outperform the performance of other state-of-the-art systems on a variety of NLP tasks. Yet unlike traditional feature-based classifiers that assign and optimize weights to varieties of hu- man interpretable features (parts-of-speech, named entities, word shapes, syntactic parse features etc) the behavior of deep learning models is much less easily interpreted. Deep learning models mainly operate on word embeddings (low-dimensional, continuous, real-valued vectors) through multi-layer neural ar- chitectures, each layer of which is characterized as an array of hidden neuron units. It is unclear how deep learning models deal with composition, imple- menting functions like negation or intensification, or combining meaning from different parts of the sen- tence, filtering away the informational chaff from the wheat, to build sentence meaning. In this paper, we explore multiple strategies to in- terpret meaning composition in neural models. We employ traditional methods like representation plot- ting, and introduce simple strategies for measur- ing how much a neural unit contributes to meaning composition, its ‘salience’ or importance using first derivatives. Visualization techniques/models represented in this work shed important light on how neural mod- els work: For example, we illustrate that LSTM’s success is due to its ability in maintaining a much sharper focus on the important key words than other models; Composition in multiple clauses works com- petitively, and that the models are able to capture neg- ative asymmetry, an important property of semantic compositionally in natural language understanding; there is sharp dimensional locality, with certain di- mensions marking negation and quantification in a manner that was surprisingly localist. Though our attempts only touch superficial points in neural mod- els, and each method has its pros and cons, together they may offer some insights into the behaviors of neural models in language based tasks, marking one initial step toward understanding how they achieve meaning composition in natural language processing. The next section describes some visualization mod- els in …

Content truncated to 500 words