Zero-shot and few-shot approaches for tokenization, tagging, and dependency parsing of Tagalog text

Zero-shot and few-shot approaches for tokenization, tagging, and dependency parsing of Tagalog text Angelina Aquino and Franz de Leon Digital Signal Processing Laboratory, Electrical and Electronics Engineering Institute University of the Philippines, Diliman, Quezon City, Philippines {angelina.aquino, franz.de.leon}@eee.upd.edu.ph Abstract The grammatical analysis of texts in any written language typically involves a num- ber of basic processing tasks, such as tok- enization, morphological tagging, and de- pendency parsing. State-of-the-art systems can achieve high accuracy on these tasks for languages with large datasets, but yield poor results for languages which have lit- tle to no annotated data. To address this is- sue for the Tagalog language, we investigate the use of alternative language resources for creating task-specific models in the absence of dependency-annotated Tagalog data. We also explore the use of word embeddings and data augmentation to improve perfor- mance when only a small amount of an- notated Tagalog data is available. We show that these zero-shot and few-shot ap- proaches yield substantial improvements on grammatical analysis of both in-domain and out-of-domain Tagalog text compared to state-of-the-art supervised baselines. 1 Introduction The grammar of a language is the set of rules and processes which govern the composition of words and sentences in that language. When conduct- ing a grammatical analysis of sentences in a given text, we typically need to perform several tasks such as tokenization, part-of-speech tagging, and syntactic parsing, each of which pertains to a dis- tinct level of grammatical structure. These three levels of analysis are illustrated in Fig. 1 as a de- pendency tree, one of the most common means of representing a sentence’s grammatical structure. With the advent of machine learning, many nat- ural language processing (NLP) pipelines can now accomplish such analysis tasks with high accu- racy for languages with large annotated datasets. However, these systems still yield poor results in a low-resource setting, for languages which have little to no available dependency-annotated data (Zeman et al., 2018). This renders the out- puts of such pipelines unsuitable for use in down- stream tasks and limits the development of more advanced technologies for these languages. This is not to say that a language which lacks annotated data is inherently a “low-resource language”—on the contrary, many such languages do in fact possess a wide range of other language resources, including literary works (e.g. books, newspapers, poetry), linguistic references (e.g. dictionaries, grammar textbooks), and the expert knowledge of those proficient with the language. Unlike annotated datasets, these resources cannot be directly fed into existing NLP pipelines as in- puts, but it is still possible to encode the infor- mation from these resources into usable formats. Given this scenario, we thus ask: 1. Can we create a pipeline for grammatical analysis without any dependency-annotated data, using only these alternative language re- sources? 2. Can we improve the performance of exist- ing pipelines by using alternative resources and methods together with a small amount of dependency-annotated data? In this work, we explore various approaches to achieve these two objectives for the …

Content truncated to 500 words