Few-Shot Pidgin Text Adaptation via Contrastive Fine-Tuning

Proceedings of the 29th International Conference on Computational Linguistics, pages 4286–4291 October 12–17, 2022. 4286 Few-Shot Pidgin Text Adaptation via Contrastive Fine-Tuning Ernie Chang1, Jesujoba O. Alabi2,4, David Ifeoluwa Adelani3,4, and Vera Demberg1 1 Department of Language Science and Technology, Saarland University, Germany 2 Inria, France 3 Spoken Language Systems (LSV), Saarland Informatics Campus, Germany 4 Masakhane NLP {cychang,vera}@coli.uni-saarland.de jesujoba.alabi@inria.fr didelani@lsv.uni-saarland.de Abstract The surging demand for multilingual dialogue systems often requires a costly labeling process for each language addition. For low resource languages, human annotators are continuously tasked with the adaptation of resource-rich lan- guage utterances for each new domain. How- ever, this prohibitive and impractical process can often be a bottleneck for low resource lan- guages that are still without proper translation systems nor parallel corpus. In particular, it is difficult to obtain task- specific low resource language annotations for the English-derived creoles (e.g. Nigerian and Cameroonian Pidgin). To address this issue, we utilize the pretrained language models i.e. BART which has shown great potential in lan- guage generation/understanding – we propose to finetune the BART model to generate utter- ances in Pidgin by leveraging the proximity of the source and target languages, and uti- lizing positive and negative examples in con- strastive training objectives. We collected and released the first parallel Pidgin-English con- versation corpus in two dialogue domains and showed that this simple and effective technique is suffice to yield impressive results for English- to-Pidgin generation, which are two closely- related languages. 1 Introduction Task-oriented dialog systems are becoming preva- lent in various daily activities such as ticket book- ing and restaurant reservations (Peng et al., 2020). In a typical task-oriented dialog system, the natural language generation (NLG) module plays a crucial role by converting semantic representations into re- sponses in natural language (Langkilde and Knight, 1998). As such, NLG plays a significant impact on the users’ experience. Unfortunately, these dialogue systems are mostly built for high resource languages such as English, which makes it less user-friendly in regions where English is not widely-spoken or is spoken differ- ently (Khalil, 2020; Yusupujiang and Ginzburg, 2021). This includes African countries such as Nigeria and Cameroon where the demand for these technologies are growing at a rapid rate (Khalil, 2020). There is then a need to adapt1/translate these existing well-developed dialogue systems in high resource languages into its target language counter- parts. However, techniques in machine translation are still under-developed as some language pairs are completely devoid of parallel corpus (Adelani et al., 2021; Chang et al., 2020). This situation is further exacerbated by the fact that the targeted NLG requires a domain-specific translation and annotation. To this end, we present a preliminary study on the domain-specific adaptation of high resource English language model into the English-related Pidgin languages consisting of the Nigerian (Naija) and Cameroonian Pidgin (Yaounde). In particu- lar, we explore on a low resource scenario where the goal is to translate the English utterance into conversation (dialogue) Pidgin sentences. This sce- nario consists of a few …

Content truncated to 500 words