Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 5970–5983, November 16–20, 2020. c⃝2020 Association for Computational Linguistics 5970 Localizing Open-Ontology QA Semantic Parsers in a Day Using Machine Translation Mehrad Moradshahi Giovanni Campagna Sina J. Semnani Silei Xu Monica S. Lam Computer Science Department Stanford University Stanford, CA, USA {mehrad,gcampagn,sinaj,silei,lam}@cs.stanford.edu Abstract We propose Semantic Parser Localizer (SPL), a toolkit that leverages Neural Machine Trans- lation (NMT) systems to localize a semantic parser for a new language. Our methodol- ogy is to (1) generate training data automat- ically in the target language by augmenting machine-translated datasets with local entities scraped from public websites, (2) add a few- shot boost of human-translated sentences and train a novel XLMR-LSTM semantic parser, and (3) test the model on natural utterances cu- rated using human translators. We assess the effectiveness of our approach by extending the current capabilities of Schema2QA, a system for English Question Answering (QA) on the open web, to 10 new languages for the restaurants and hotels do- mains. Our models achieve an overall test ac- curacy ranging between 61% and 69% for the hotels domain and between 64% and 78% for restaurants domain, which compares favorably to 69% and 80% obtained for English parser trained on gold English data and a few ex- amples from validation set. We show our ap- proach outperforms the previous state-of-the- art methodology by more than 30% for hotels and 40% for restaurants with localized ontolo- gies for the subset of languages tested. Our methodology enables any software devel- oper to add a new language capability to a QA system for a new domain, leveraging machine translation, in less than 24 hours. Our code is released open-source.1 1 Introduction Localization is an important step in software or website development for reaching an international audience in their native language. Localization is usually done through professional services that can translate text strings quickly into a wide variety of 1https://github.com/stanford-oval/SPL languages. As conversational agents are increas- ingly used as the new interface, how do we localize them to other languages efficiently? The focus of this paper is on question answering systems that use semantic parsing, where natural language is translated into a formal, executable representation (such as SQL). Semantic parsing typically requires a large amount of training data, which must be annotated by an expert familiar with both the natural language of interest and the formal language. The cost of acquiring such a dataset is prohibitively expensive. For English, previous work has shown it is possi- ble to bootstrap a semantic parser without massive amount of manual annotation, by using a large, hand-curated grammar of natural language (Wang et al., 2015; Xu et al., 2020a). This approach is ex- pensive to replicate for all languages, due to the ef- fort and expertise required to build such a grammar. Hence, we investigate the question: Can we lever- age previous work on English semantic parsers for other languages by using machine translation? And in particular, can …

Content truncated to 500 words