NLP Service APIs and Models for Efficient Registration of New Clients

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4007–4012 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4007 NLP Service APIs and Models for Efficient Registration of New Clients Sahil Shah1, Vihari Piratla1, Soumen Chakrabarti1, Sunita Sarawagi1 1Department of Computer Science, Indian Institute of Technology, Bombay, India sahilshah00199@gmail.com, {vihari,soumen,sunita}@cse.iitb.ac.in Abstract State-of-the-art NLP inference uses enormous neural architectures and models trained for GPU-months, well beyond the reach of most consumers of NLP. This has led to one-size- fits-all public API-based NLP service models by major AI companies, serving large numbers of clients. Neither (hardware deficient) clients nor (heavily subscribed) servers can afford tra- ditional fine tuning. Many clients own little or no labeled data. We initiate a study of adap- tation of centralized NLP services to clients, and present one practical and lightweight ap- proach. Each client uses an unsupervised, corpus-based sketch to register to the service. The server uses an auxiliary network to map the sketch to an abstract vector representation, which then informs the main labeling network. When a new client registers with its sketch, it gets immediate accuracy benefits. We demon- strate the success of the proposed architecture using sentiment labeling, NER, and predictive language modeling. 1 Introduction State-of-the-art NLP uses large neural networks with billions of parameters, enormous training data, and intensive optimization over weeks of GPU- time, causing more carbon emission than a car over its lifetime (Strubell et al., 2019). Such training prowess is (mercifully) out of reach for most users of NLP methods. Recognizing this, large AI com- panies have launched NLP cloud services1 and also provided trained models for download and fine tun- ing. But many clients have too little data or hard- ware for fine tuning massive networks. Neither can the service be expected to fine-tune for each client. Distributional mismatch between the giant general-purpose corpus used to train the central ser- vice and the corpus from which a client’s instances 1Google NLP, Microsoft Azure, IBM Watson arise leads to lower accuracy. A common source of trouble is mismatch of word salience (Paik, 2013) between client and server corpora (Ruder, 2019). In this respect, our setting also presents a new op- portunity. Clients are numerous and form natural clusters, e.g., healthcare, sports, politics. We want the service to exploit commonalities in existing client clusters, without explicitly supervising this space, and provide some level of generalization to new clients without re-training or fine-tuning. In response to the above challenges and con- straints, we initiate an investigation of practical protocols for lightweight client adaptation of NLP services. We propose a system, KYC (“Know Your Client”), in which each client registers with the service using a simple sketch derived from its (un- labeled) corpus. The service network takes the sketch as additional input with each instance later submitted by the client. The service provides accu- racy benefits to new clients immediately. What form can a client sketch take? How should the service network incorporate it? While this will depend on the …

Content truncated to 500 words