Active Learning for Multilingual Semantic Parser

Active Learning for Multilingual Semantic Parser Zhuang Li, Gholamreza Haffari Openstream.AI {zhuang.li,reza.haffari}@openstream.com Abstract Current multilingual semantic parsing (MSP) datasets are almost all collected by translat- ing the utterances in the existing datasets from the resource-rich language to the target lan- guage. However, manual translation is costly. To reduce the translation effort, this paper pro- poses the first active learning procedure for MSP (AL-MSP). AL-MSP selects only a sub- set from the existing datasets to be translated. We also propose a novel selection method that prioritizes the examples diversifying the logi- cal form structures with more lexical choices, and a novel hyperparameter tuning method that needs no extra annotation cost. Our experi- ments show that AL-MSP significantly reduces translation costs with ideal selection methods. Our selection method with proper hyperparam- eters yields better parsing performance than the other baselines on two multilingual datasets. 1 Introduction Multilingual semantic parsing converts multilin- gual natural language utterances into logical forms (LFs) using a single model. However, there is a severe data imbalance among the MSP datasets. Currently, most semantic parsing datasets are in En- glish, while only a limited number of non-English datasets exist. To tackle the data imbalance issue, almost all current efforts build MSP datasets by translating utterances in the existing datasets from the resource-rich language (e.g. English) into other languages (Duong et al., 2017; Li et al., 2021a). However, manual translation is slow and labori- ous. In such cases, active learning is an excellent solution to lower the translation cost. Active learning (AL) is a family of methods that collects training data when the annotation bud- gets are limited (Lewis and Catlett, 1994). Our work proposes the first active learning approach for MSP. Compared to translating the full dataset, AL-MSP aims to select only a subset from the ex- isting dataset to be translated, which significantly reduces the translation cost. We further study which examples AL-MSP should select to optimize multilingual parsing per- formance. Oren et al. (2021) demonstrated that a training set with diverse LF structures signifi- cantly enhances compositional generalization of the parsers. Furthermore, our experiments show that the examples with LFs aligned with more di- versified lexical variants in the training set consid- erably improve the performance of multilingual parsing during AL. Motivated by both, we propose a novel strategy for selecting the instances which include diversified LF structures with more lexical choices. Our selection method yields better parsing performance than the other baselines. By translat- ing just 32% of all examples, the parser achieves comparable performance on multilingual GEO- QUERY and NLMAP as translating full datasets. Prior works obtain the hyperparameters of the AL methods by either copying configurations from comparable settings or tuning the hyperparameters on the seed evaluation data (Duong et al., 2018). However, the former method is not suitable as our AL setting is unique, whereas the second method requires extra annotation costs. In this work, we provide a cost-free method for our AL scenario for obtaining optimal hyperparameters. Our contributions are i) the …

Content truncated to 500 words