CNM: An Interpretable Complex-valued Network for Matching

Proceedings of NAACL-HLT 2019, pages 4139–4148 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 4139 CNM: An Interpretable Complex-valued Network for Matching Qiuchi Li ∗ University of Padua Padua, Italy qiuchili@dei.unipd.it Benyou Wang ∗ University of Padua Padua, Italy wang@dei.unipd.it Massimo Melucci † University of Padua Padua, Italy melo@dei.unipd.it Abstract This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this frame- work unifies different linguistic units in a sin- gle complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued net- work is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network ad- mits interpretations to explicit physical mean- ings. The proposed complex-valued network for matching (CNM)1 achieves comparable performances to strong CNN and RNN base- lines on two benchmarking question answer- ing (QA) datasets. 1 Introduction There is a growing concern on the interpretabil- ity of neural networks. Along with the increasing power of neural networks comes the challenge of interpreting the numerical representation of net- work components into human-understandable lan- guage. Lipton (2018) points out two important factors for a model to be interpretable, namely post-hoc interpretability and transparency. The former refers to explanations of why a model works after it is executed, while the latter concerns self-explainability of components through some mechanisms in the designing phase of the model. We seek inspirations from quantum physics to build transparent and post-hoc interpretable net- works for modeling human language. The emerg- ing research field of cognition suggests that there exist quantum-like phenomena in human cogni- tion (Aerts and Sozzo, 2014), especially language understanding (Bruza et al., 2008). Intuitively, a ∗Equal Contribution †Corresponding Author 1https://github.com/wabyking/qnn.git sentence can be treated as a physical system with multiple words (like particles), and these words are usually polysemous (superposed) and corre- lated (entangled) with each other. Motivated by these existing works, we aim to investigate the fol- lowing Research Question (RQ). RQ1: Is it possible to model human lan- guage with the mathematical framework of quan- tum physics? Towards this question, we build a novel quantum-theoretic framework for modeling lan- guage, in an attempt to capture the quantum- ness in the cognitive aspect of human language. The framework models different linguistic units as quantum states with the adoption of quan- tum probability (QP), which is the mathematical framework of quantum physics that models uncer- tainly on a uniform Semantic Hilbert Space (SHS). Complex values are crucial in the mathematical framework of characterizing quantum physics. In order to preserve physical properties, the linguis- tic units have to be represented as complex vectors or matrices. This naturally gives rise to another re- search question: RQ2: Can we benefit from the complex-valued representation of human language in a real natu- ral language processing (NLP) scenario? To this end, we formulate a linguistic unit as a complex-valued vector, and link its length and di- rection to different physical …

Content truncated to 500 words