A Unifying Theory of Transition-based and Sequence Labeling Parsing

Proceedings of the 28th International Conference on Computational Linguistics, pages 26–38 Barcelona, Spain (Online), December 8-13, 2020 26 Building Hierarchically Disentangled Language Models for Text Generation with Named Entities Yash Agarwal* NSUT, New Delhi yasha.co.17@nsit.net.in Ganesh Bagler IIIT-Delhi, New Delhi bagler@iiitd.ac.in Devansh Batra* NSUT, New Delhi devanshb.it.17@nsit.net.in Abstract Named entities pose a unique challenge to traditional methods of language modeling. While several domains are characterised with a high proportion of named entities, the occurrence of specific entities varies widely. Cooking recipes, for example, contain a lot of named entities — viz. ingredients, cooking techniques (also called processes), and utensils. However, some ingre- dients occur frequently within the instructions while most occur rarely. In this paper, we build upon the previous work done on language models developed for text with named entities by in- troducing a Hierarchically Disentangled Model. Training is divided into multiple branches with each branch producing a model with overlapping subsets of vocabulary. We found the existing datasets insufficient to accurately judge the performance of the model. Hence, we have curated 158,473 cooking recipes from several publicly available online sources. To reliably derive the entities within this corpus, we employ a combination of Named Entity Recognition (NER) as well as an unsupervised method of interpretation using dependency parsing and POS tagging, followed by a further cleaning of the dataset. This unsupervised interpretation models instruc- tions as action graphs and is specific to the corpus of cooking recipes, unlike NER which is a general method applicable to all corpora. To delve into the utility of our language model, we ap- ply it to tasks such as graph-to-text generation and ingredients-to-recipe generation, comparing it to previous state-of-the-art baselines. We make our dataset (including annotations and processed action graphs) available for use, considering their potential use cases for language modeling and text generation research. 1 Introduction Language Modeling is an important task in Natural Language Processing with several applications (Wise- man et al., 2017) including auto complete (Arnold et al., 2017) among others. A key application of language modeling is text generation (Semeniuta et al., 2017). Recent advances in language models have led to impressive generation of programming code in languages such as Java and C (Hindle et al., 2016; Yin and Neubig, 2017; Hellendoorn and Devanbu, 2017; Rabinovich et al., 2017). Similarly, efforts have been made towards application of language models in the context of cooking recipes (Parvez et al., 2018; Kiddon et al., 2016). Both these tasks, recipe and code generation, deal with corpora heavily populated with named entities. In case of cooking recipes these entities are thousands of ingredients used in recipes, hundreds of cooking processes which can be applied upon them and tens of utensils used across the different cuisines. These numerous entities immensely increase the vocabulary of the language model. Typically, the vocabulary of corpora of cooking recipes is of the order of tens of thousands of tokens. The vocabulary of the corpus we prepared (more details will follow later in the manuscript) has This work …

Content truncated to 500 words