On-device Structured and Context Partitioned Projection Networks

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3784–3793 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 3784 On-device Structured and Context Partitioned Projection Networks Sujith Ravi Google Research Mountain View, CA, USA sravi@google.com Zornitsa Kozareva Google Mountain View, CA, USA zornitsa@kozareva.com Abstract A challenging problem in on-device text clas- sification is to build highly accurate neural models that can fit in small memory foot- print and have low latency. To address this challenge, we propose an on-device neural network SGNN++ which dynamically learns compact projection vectors from raw text us- ing structured and context-dependent partition projections. We show that this results in ac- celerated inference and performance improve- ments. We conduct extensive evaluation on multiple conversational tasks and languages such as En- glish, Japanese, Spanish and French. Our SGNN++ model significantly outperforms all baselines, improves upon existing on-device neural models and even surpasses RNN, CNN and BiLSTM models on dialog act and in- tent prediction. Through a series of ablation studies we show the impact of the partitioned projections and structured information lead- ing to 10% improvement. We study the im- pact of the model size on accuracy and intro- duce quantization-aware training for SGNN++ to further reduce the model size while preserv- ing the same quality. Finally, we show fast in- ference on mobile phones. 1 Introduction Over the last years, the usage of conversational as- sistants has become extremely popular. On a daily basis, people request weather information, check calendar appointments, perform calls. Large part of the conversational and natural language under- standing happens on the server side and then ful- filled resulting in response delays, inconsistent ex- perience and privacy concerns. Therefore, there is a huge demand for developing on-device natu- ral language models that work entirely on-device such as mobile phones, tablets, watches and any internet of things (IoT) devices. On-device com- putation can circumvent the latency delays, can in- crease the user privacy and further enable new ca- pabilities for real time interaction. One way to develop on-device natural lan- guage understanding is to leverage the power of deep neural networks, which over the years have shown tremendous progress and have improved upon state-of-the-art machine learning methods in Natural Language Processing (NLP) (Sutskever et al., 2014), Speech (Hinton et al., 2012) and Vi- sion (Krizhevsky et al., 2012). These advance- ments were byproducts of the availability of large amounts of data and high performance computing, enabling the development of more complex and robust neural network architectures. However, de- spite their success, yet it remains challenging to deploy deep networks on-device such as mobile phone, smart watch and IoT. The limited memory and computation power combined with the need of fast latency require the development of novel on-device neural networks. Inspired by (Ravi and Kozareva, 2018), we pro- pose a novel on-device neural network (SGNN++ ) that uses joint structured (word+character) infor- mation and context partitioned projections to learn robust models for short text classification. …

Content truncated to 500 words