Dense Template Retrieval for Customer Support

Proceedings of the 29th International Conference on Computational Linguistics, pages 1106–1115 October 12–17, 2022. 1106 Dense Template Retrieval for Customer Support Tiago Mesquita∗ Zendesk Lisbon, Portugal tiago.mesquita@zendesk.com Bruno Martins INESC-ID & LUMLIS (Lisbon ELLIS Unit) IST, University of Lisbon Lisbon, Portugal bruno.g.martins@inesc-id.pt Mariana Almeida Zendesk Lisbon, Portugal mariana.almeida@zendesk.com Abstract Templated answers are used extensively in cus- tomer support scenarios, providing an efficient way to cover a plethora of topics, with an easily maintainable collection of templates. However, the number of templates is often too high for an agent to manually search. Automatically sug- gesting the correct template for a given ques- tion can thus improve the service efficiency, reducing costs and leading to a better customer satisfaction. In this work, we propose a dense retrieval framework for the customer support scenario, adapting a standard in-batch nega- tives technique to support unpaired sampling of queries and templates. We also propose a novel loss that extends the typical query-centric similarity, exploiting other similarity relations in the training data. Experiments show that our approach achieves considerable improve- ments, in terms of performance and training speed, over more standard dense retrieval meth- ods. This includes methods such as DPR, and also ablated versions of the proposed approach. 1 Introduction A very common practice to make customer support more efficient is the use of templates for replies. The templates of replies are designed by customer support administrators to systematise the reply to frequent requests. Then, given a customer’s request, an agent can pick a response from within a collec- tion of predefined templates, this way saving time when replying to repetitive questions. Besides im- proving the throughput of human agents, the use of templates also assures uniformity in the handling of different customers, as requests with the same un- derlying problem should be handled with the same type of reply. However, customer support centers can have hundreds of templates, and finding the best template for a question is not an easy task, par- ticularly for unexperienced agents. Automatically ∗This work was developed at Cleverly and Zendesk, under the context of Tiago Mesquita’s Master of Science (M.Sc.) Thesis at IST, University of Lisbon. q0 q0 q1 q1 q2 q2 t0 t1 t2 (a) query-template q0 q0 q1 q1 q2 q2 t0 t1 t2 (b) query-template, query- query and template-template Figure 1: Illustration of 6 query representations {q0, q0, q1, q1, q2, q2}, together with respective template representations {t0, t1, t2} with ti answering qi, after enforcing different similarity relations. The distances between points represent dissimilarity, and dashed lines and circumferences represent similarity relations, whilst the dotted lines represent dissimilarity relations. sorting and suggesting customer support templates (Bonatti et al., 2016; Yang and Kwok, 2012; Snei- ders et al., 2016; Rei, 2019) can facilitate agent’s work, reducing reply times, accelerating the learn- ing curve of new agents, helping agents to focus on more added valued tasks, and overall providing a better customer support at reduced costs. The recent advances in large pre-trained lan- guage models (Devlin et …

Content truncated to 500 words