Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3816–3831 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3816 Total Recall: a Customized Continual Learning Method for Neural Semantic Parsers Zhuang Li, Lizhen Qu∗, Gholamreza Haffari Faculty of Information Technology Monash University firstname.lastname@monash.edu Abstract This paper investigates continual learning for semantic parsing. In this setting, a neural se- mantic parser learns tasks sequentially with- out accessing full training data from previous tasks. Direct application of the SOTA contin- ual learning algorithms to this problem fails to achieve comparable performance with re- training models with all seen tasks, because they have not considered the special prop- erties of structured outputs, yielded by se- mantic parsers. Therefore, we propose TO- TAL RECALL, a continual learning method de- signed for neural semantic parsers from two aspects: i) a sampling method for memory replay that diversifies logical form templates and balances distributions of parse actions in a memory; ii) a two-stage training method that significantly improves generalization capabil- ity of the parsers across tasks. We conduct extensive experiments to study the research problems involved in continual semantic pars- ing, and demonstrate that a neural semantic parser trained with TOTAL RECALL achieves superior performance than the one trained di- rectly with the SOTA continual learning algo- rithms, and achieve a 3-6 times speedup com- pared to retraining from scratch. Code and datasets are available at: https://github. com/zhuang-li/cl_nsp. 1 Introduction In the recent market research report published by MarketsandMarkets (INC., 2020), it is estimated that the smart speaker market is expected to grow from USD 7.1 billion in 2020 to USD 15.6 bil- lion by 2025. Commercial smart speakers, such as Alexa and Google assistant, often need to trans- late users’ commands and questions into actions. Therefore, semantic parsers are widely adopted in dialogue systems to map natural language (NL) ut- terances to executable programs or logical forms ∗corresponding author (LFs) (Damonte et al., 2019; Rongali et al., 2020). Due to the increasing popularity of such speakers, software developers have implemented a large vol- ume of skills for them and the number of new skills grow quickly every year. For example, as of 2020, the number of Alexa skills exceeds 100,000 and 24 new skills are introduced per day in 2020 (KIN- SELLA, 2020). Although machine learning-based semantic parsers achieve the state-of-the-art perfor- mance, they face the following challenges due to the fast growing number of tasks. Given new tasks, one common practice is to re- train the parser from scratch on the training data of all seen tasks. However, it is both economically and computationally expensive to re-train seman- tic parsers because of a fast-growing number of new tasks (Lialin et al., 2020). To achieve its op- timal performance, training a deep model on all 8 tasks of NLMap (Lawrence and Riezler, 2018) takes approximately 14 times longer than training the same model on one of those tasks. In practice, the cost of repeated re-training for a commercial smart speaker is …

Content truncated to 500 words