Semantic Parsing for Technical Support Questions

Proceedings of the 27th International Conference on Computational Linguistics, pages 3251–3259 Santa Fe, New Mexico, USA, August 20-26, 2018. 3251 Semantic Parsing of Technical Support Questions Abhirut Gupta, Anupama Ray, Gargi Dasgupta, Gautam Singh, Pooja Aggarwal and Prateeti Mohapatra IBM Research AI {abhirutgupta, anupamar, gaargidasgupta, gautamsi, aggarwal.pooja, pramoh01}@in.ibm.com Abstract Technical support problems are very complex. In contrast to regular web queries (that contain few keywords) or factoid questions (which are a few sentences), these problems usually include attributes like a detailed description of what is failing (symptom), steps taken in an effort to reme- diate the failure (activity), and sometimes a specific request or ask (intent). Automating support is the task of automatically providing answers to these problems given a corpus of solution doc- uments. Traditional approaches to this task rely on information retrieval and are keyword based; looking for keyword overlap between the question and solution documents and ignoring these attributes. We present an approach for semantic parsing of technical questions that uses gram- matical structure to extract these attributes as a baseline, and a CRF based model that can improve performance considerably in the presence of annotated data for training. We also demonstrate that combined with reasoning, these attributes help outperform retrieval baselines. 1 Introduction Technical support services involve enterprises providing after-sales support to users of technology prod- ucts. Traditionally, this entails employing a large number of human agents to manually diagnose, trou- bleshoot, and resolve customer problems. However, with ever increasing variety of products and the exponential growth of users, this model doesn’t scale. Costs of hiring and training additional talent to keep up with this growth is a huge overhead. There is a clear incentive to automate this process, reduc- ing manual effort and resolution times. Automating technical support refers to the task of automatically understanding user questions, performing diagnosis and troubleshooting, either remotely or in conversa- tion with the user, and providing resolutions. Given the massive potential impact, it is no surprise that the problem draws attention and investment from many large corporations as well as startups (Flinders, 2015), and (Dhoolia et al., 2017). Automating technical support has a very broad goal, and involves solving various sub-problems like document representation (Yang et al., 2017), identifying and extracting procedures, log analysis, and goal based dialoging (Bordes and Weston, 2016). Each of these problems has a plethora of daunting research challenges that remain largely unsolved. In this paper, we restrict our focus to the problem of understanding questions in technical support and retrieving relevant resolution documents from a corpus of support content. While a complete automation system would also tackle the challenging problem of identifying the troubleshooting/diagnostic procedure from these documents, we assume that a returned “relevant” document contains procedures required for resolving the problem. As identified in (Gupta et al., 2017) technical support problems are complex, and involve various at- tributes like failing entities, a detailed description of the problem (symptom), steps taken in an effort to remediate the failure (activity), and sometimes a specific request …

Content truncated to 500 words