Unsupervised Slot Schema Induction for Task-oriented Dialog

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1174 - 1193 July 10-15, 2022 ©2022 Association for Computational Linguistics Unsupervised Slot Schema Induction for Task-oriented Dialog Dian Yu1∗, Mingqiu Wang2, Yuan Cao2, Izhak Shafran2, Laurent El Shafey2, Hagen Soltau2 1 University of California, Davis 2 Google Research dianyu@ucdavis.edu {mingqiuwang, yuancao, izhak, shafey, soltau}@google.com Abstract Carefully-designed schemas describing how to collect and annotate dialog corpora are a pre- requisite towards building task-oriented dialog systems. In practical applications, manually designing schemas can be error-prone, labori- ous, iterative, and slow, especially when the schema is complicated. To alleviate this expen- sive and time consuming process, we propose an unsupervised approach for slot schema in- duction from unlabeled dialog corpora. Lever- aging in-domain language models and unsu- pervised parsing structures, our data-driven ap- proach extracts candidate slots without con- straints, followed by coarse-to-fine clustering to induce slot types. We compare our method against several strong supervised baselines, and show significant performance improvement in slot schema induction on MultiWoz and SGD datasets. We also demonstrate the effective- ness of induced schemas on downstream ap- plications including dialog state tracking and response generation. 1 Introduction Defining task-specific schemas, including intents and arguments, is the first step of building a task- oriented dialog (TOD) system. In real-world appli- cations such as call centers, we may have abundant conversation logs from real users and system assis- tants without annotation. To build an effective sys- tem, experts need to study thousands of conversa- tions, find relevant phrases, manually group phrases into concepts, and iteratively build the schema to cover use cases. The schema is then used to an- notate belief states and train models. This pro- cess is labor-intensive, error-prone, expensive, and slow (Eric et al., 2020; Zang et al., 2020; Min et al., 2020; Yu and Yu, 2021). As a prerequisite, it hin- ders quick deployment for new domains and tasks. We therefore are interested in developing automatic ∗work done during internship at Google Research Can I have more information for the train you’re needing? I am leaving from Cambridge and going to Norwich. I also need a train. The train should leave after 16:15 and leave on sunday. … … I want to book an expensive Italian restaurant at 12 pm. after 16:15 before 9:30 12:30 … cheap expensive moderate … 12 pm 7:20 9 … Span extraction Multi-step clustering train leaveat hotel pricerange restaurant booktime Figure 1: Overview of slot schema induction from raw conversations. We use a bottom-up representation level distance function derived from pre-trained LMs (com- bined with PCFG structure) to extract informative can- didate phrases such as “after 16:15” and “expensive”. The spans are subsequently clustered through multiple stages to form coarse to fine categories. The ground truth mapping is shown on the right (such as “train leaveat”). schema induction methods in this work to create the ontology1 from conversations for TOD tasks. Most existing approaches for slot schema induc- tion …

Content truncated to 500 words