Dual Inference for Improving Language Understanding and Generation

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4930–4936 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4930 Dual Inference for Improving Language Understanding and Generation Shang-Yu Su⋆ Yung-Sung Chuang⋆ Yun-Nung Chen National Taiwan University, Taipei, Taiwan {b05901033,f05921117}@ntu.edu.tw y.v.chen@ieee.org Abstract Natural language understanding (NLU) and Natural language generation (NLG) tasks hold a strong dual relationship, where NLU aims at predicting semantic labels based on natural language utterances and NLG does the oppo- site. The prior work mainly focused on ex- ploiting the duality in model training in order to obtain the models with better performance. However, regarding the fast-growing scale of models in the current NLP area, sometimes we may have difficulty retraining whole NLU and NLG models. To better address the issue, this paper proposes to leverage the duality in the inference stage without the need of retraining. The experiments on three benchmark datasets demonstrate the effectiveness of the proposed method in both NLU and NLG, providing the great potential of practical usage. 1 1 Introduction Various tasks, though different in their goals and formations, are usually not independent and yield diverse relationships between each other within each domain. It has been found that many tasks come with a dual form, where we could directly swap the input and the target of a task to formulate into another task. Such structural duality emerges as one of the important relationship for further in- vestigation, which has been utilized in many tasks including machine translation (Wu et al., 2016), speech recognition and synthesis (Tjandra et al., 2017), and so on. Previous work first exploited the duality of the task pairs and proposed supervised (Xia et al., 2017) and unsupervised (reinforcement learning) (He et al., 2016) learning frameworks in machine translation. The recent studies magnified the importance of the duality by revealing exploita- tion of it could boost the learning for both tasks. 1The source code and data are available at https:// github.com/MiuLab/DuaLUG. ⋆The first two authors contributed to this paper equally. Natural language understanding (NLU) (Tur and De Mori, 2011; Hakkani-T¨ur et al., 2016) and nat- ural language generation (NLG) (Wen et al., 2015; Su et al., 2018) are two major components in mod- ular conversational systems, where NLU extracts core semantic concepts from the given utterances, and NLG constructs the associated sentences based on the given semantic representations. Su et al. (2019) was the first attempt that leveraged the duality in dialogue modeling and employed the dual supervised learning framework for training NLU and NLG. Furthermore, Su et al. (2020) pro- posed a joint learning framework that can train two modules seamlessly towards the potential of unsupervised NLU and NLG. Recently, Zhu et al. (2020) proposed a semi-supervised framework to learn NLU with an auxiliary generation model for pseudo-labeling to make use of unlabeled data. Despite the effectiveness showed by the prior work, they all focused on leveraging the duality in the training process to obtain powerful NLU and NLG models. However, there has been little …

Content truncated to 500 words