Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenario

Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7354 - 7370 December 7-11, 2022 ©2022 Association for Computational Linguistics Counterfactual Recipe Generation: Exploring Compositional Generalization in a Realistic Scenario Xiao Liu1, Yansong Feng1,2∗, Jizhi Tang3, Chengang Hu1 and Dongyan Zhao1,4,5 1Wangxuan Institute of Computer Technology, Peking University 2The MOE Key Laboratory of Computational Linguistics, Peking University 3Baidu Inc., Beijing, China 4Beijing Institute for General Artificial Intelligence 5State Key Laboratory of Media Convergence Production Technology and Systems {lxlisa,fengyansong,hcg,zhaody}@pku.edu.cn tangjizhi@baidu.com Abstract People can acquire knowledge in an unsuper- vised manner by reading, and compose the knowledge to make novel combinations. In this paper, we investigate whether pretrained language models can perform compositional generalization in a realistic setting: recipe gen- eration. We design the counterfactual recipe generation task, which asks models to modify a base recipe according to the change of an ingredient. This task requires compositional generalization at two levels: the surface level of incorporating the new ingredient into the base recipe, and the deeper level of adjusting actions related to the changing ingredient. We collect a large-scale recipe dataset in Chinese for models to learn culinary knowledge, and a subset of action-level fine-grained annotations for evaluation. We finetune pretrained language models on the recipe corpus, and use unsuper- vised counterfactual generation methods to gen- erate modified recipes. Results show that exist- ing models have difficulties in modifying the ingredients while preserving the original text style, and often miss actions that need to be adjusted. Although pretrained language mod- els can generate fluent recipe texts, they fail to truly learn and use the culinary knowledge in a compositional way. Code and data are available at https://github.com/xxxiaol/ counterfactual-recipe-generation. 1 Introduction Reading is an effective way to gain knowledge. When people read, mental processes like structured information extraction and rule discovery go on in our brains (Gibson and Levin, 1975). In the case of cooking, we read recipes of various dishes, gain knowledge of ingredients and flavors, and compose them to cook other dishes. 1 ∗Corresponding author. 1We provide explanations of concepts related to recipes in Table 1 for better understanding. Red-braised: add soy sauce, simmer over gentle heat, … Crucian Carp: make diagonal cuts on each side of the fish, … …… Red-braised Crucian Carp Red-braised Pork 1. … 2. … Crucian Carp Soup 1. … 2. … Red-braised Beef Brisket 1. … 2. … Pan-fried Crucian Carp 1. … 2. … Red-braised Pork 1. … 2. … Crucian Carp Soup 1. … 2. … Red-braised Beef Brisket 1. … 2. … Pan-fried Crucian Carp 1. … 2. … Figure 1: An example of the recipe learning process of human for the dish red-braised crucian carp. This process involves knowledge acquisition and composition. As shown in Figure 1, when people read recipes, they distill the knowledge of flavors and ingredients, like soy sauce is usually used to get red-braised flavor and people often make diagonal cuts on fish to better marinate. People can then cook new dishes like …

Content truncated to 500 words