Natural Language Model Re-usability for Scaling to Different Domains

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2071–2076, Austin, Texas, November 1-5, 2016. c⃝2016 Association for Computational Linguistics Natural Language Model Re-usability for Scaling to Different Domains Young-Bum Kim, Alexandre Rochette and Ruhi Sarikaya Microsoft Corporation, Redmond, WA ybkim,alrochet,ruhi.sarikaya@microsoft.com Abstract Natural language understanding is the core of the human computer interactions. How- ever, building new domains and tasks that need a separate set of models is a bottle- neck for scaling to a large number of do- mains and experiences. In this paper, we propose a practical technique that addresses this issue in a web-scale language understand- ing system: Microsoft’s personal digital as- sistant Cortana. The proposed technique uses a constrained decoding method with a uni- versal slot tagging model sharing the same schema as the collection of slot taggers built for each domain. The proposed approach al- lows reusing of slots across different domains and tasks while achieving virtually the same performance as those slot taggers trained per domain fashion. 1 Introduction Recently there has been tremendous investment into the personal digital assistants by big technology companies (Sarikaya, 2015; Sarikaya et al., 2016). Apple’s SIRI, Google Now, Microsoft’s Cortana and Amazon’s Alexa are examples of such systems. Nat- ural language understanding (Gupta et al., 2006; Tur and De Mori, 2011)is at the core of these systems providing natural communication between the user and the agent. These systems support a number of scenarios including creating reminders, setting up alarms, note taking, scheduling meetings, finding and consuming entertainment (i.e. movie, music, games), finding places of interest and getting driv- ing directions to them. The domains supported by these systems are on the order of tens (not in hun- dreds) and adding new domains and experiences is a scaling problem that has not been solved yet (Tur, 2006; Jeong and Lee, 2009; El-Kahky et al., 2014; Kim et al., 2015d). The primary reason behind this is that each do- main requires potentially a new schema, intents and slots extracted from the natural language query. That, in turn requires collecting and annotating new data, which is the most expensive step in terms of time and money, and building a new set of domain specific models to support the new domains and sce- narios. Slot modeling in particular is the most de- manding component in terms of the difficulty of an- notation and modeling. In this study, we propose a new approach that re- duces the cost of scaling natural language under- standing to a large number of domains and expe- riences without significantly sacrificing the accu- racy. The approach consists of a universal slot tag- ging method and a runtime technique called con- strained decoding that performs the decoding ac- cording a specific schema. The proposed approach, heavily enables reusing existing slot modeling data that are collected and annotated for potentially dif- ferent domains and applications, for building new domains and applications. The new domains can be expressed in terms of existing semantic schema. The rest of …

Content truncated to 500 words