A Neural Network Component for Knowledge-Based Semantic Representations of Text

Proceedings of Recent Advances in Natural Language Processing, pages 904–911, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_105 904 A Neural Network Component for Knowledge-Based Semantic Representations of Text Alejandro Piad-Morffis1, Rafael Mu˜noz2, Yudivi´an Almeida-Cruz1, Yoan Guti´errez3, Suilan Estevez-Velarde1, and Andr´es Montoyo2 1School of Math and Computer Science, University of Havana, Cuba {sestevez,yudy,apiad}@matcom.uh.cu 2Department of Languages and Computing Systems, University of Alicante, Spain 3U.I. for Computer Research (IUII), University of Alicante, Spain {montoyo,ygutierrez,rafael}@dlsi.ua.es Abstract This paper presents Semantic Neural Net- works (SNNs), a knowledge-aware com- ponent based on deep learning. SNNs can be trained to encode explicit seman- tic knowledge from an arbitrary knowl- edge base, and can subsequently be com- bined with other deep learning architec- tures. At prediction time, SNNs provide a semantic encoding extracted from the in- put data, which can be exploited by other neural network components to build ex- tended representation models that can face alternative problems. The SNN architec- ture is defined in terms of the concepts and relations present in a knowledge base. Based on this architecture, a training pro- cedure is developed. Finally, an experi- mental setup is presented to illustrate the behaviour and performance of a SNN for a specific NLP problem, in this case, opin- ion mining for the classification of movie reviews. 1 Introduction In recent years, the increase in the volume of avail- able data has provided both new techniques and new challenges for discovering relevant knowl- edge. The huge amount of information produced daily makes it impossible for humans to manu- ally build organized representations of all this in- formation. On the other hand, the surplus of in- formation enables the design of statistical learn- ing techniques which scale better with large data. Deep learning approaches have successfully ob- tained state-of-the-art results for many learning problems, from image recognition to natural lan- guage parsing. Instead of handcrafted representa- tions designed by experts, deep learning automati- cally builds representations from raw data that are suitable for a given machine learning problem. When deep learning is used, we can often iden- tify structures inside a neural network, which can be explained as high-level concepts that are au- tomatically discovered during the learning pro- cess (Bengio, 2012). For instance, in the im- age recognition domain, often internal neurons or groups of neurons can be identified to recognize common visual features such as textures or pat- terns or even specific objects (Le, 2013). Hence, the neural network is able to build high-level con- cepts from low-level input (i.e., pixels). This abil- ity to discover higher-level features is one of the main strengths of deep learning and representa- tional learning in general (Bengio, 2012). How- ever, one of the main challenges of deep learn- ing is interpreting the internal representations of a neural network in terms that can be understood by humans and mapped to clearly defined domain concepts (Montavon et al., 2017). To address this challenge, our proposal is to“persuade” a neural network to build repre- sentations that can be interpreted in terms …

Content truncated to 500 words