BRCC and SentiBahasaRojak: The First Bahasa Rojak Corpus for Pretraining and Sentiment Analysis Dataset

Proceedings of the 29th International Conference on Computational Linguistics, pages 4418–4428 October 12–17, 2022. 4418 BRCC and SentiBahasaRojak: The First Bahasa Rojak Corpus for Pretraining and Sentiment Analysis Dataset Nanda Putri Romadhona† Sin-En Lu† Bo-Han Lu† Richard Tzong-Han Tsai†‡∗ †Department of Computer Science and Information Engineering, National Central University, Taiwan ‡Center for GIS, Research Center for Humanities and Social Sciences, Academia Sinica, Taiwan nandadona61@gmail.com {alznn, lu110522028, thtsai}@g.ncu.edu.tw Abstract Code-mixing refers to the mixed use of multi- ple languages. It is prevalent in multilingual societies and is also one of the most challeng- ing natural language processing tasks. In this paper, we study Bahasa Rojak, a dialect popu- lar in Malaysia that consists of English, Malay, and Chinese. Aiming to establish a model to deal with the code-mixing phenomena of Ba- hasa Rojak, we use data augmentation to auto- matically construct the first Bahasa Rojak cor- pus for pre-training language models, which we name the Bahasa Rojak Crawled Corpus (BRCC). We also develop a new pre-trained model called "Mixed XLM". The model can tag the language of the input token automati- cally to process code-mixing input. Finally, to test the effectiveness of the Mixed XLM model pre-trained on BRCC for social media scenar- ios where code-mixing is found frequently, we compile a new Bahasa Rojak sentiment analy- sis dataset, SentiBahasaRojak1, with a Kappa value of 0.77. 1 Introduction Code-mixing is common in multilingual societies (Bukhari et al., 2015). People tend to use one primary language for grammar and scripting (Lal et al., 2019), and other languages as auxiliary. Code-mixing is commonly found on social media, such as Facebook, Twitter, or any other microblog services. Malaysia reflects a multilingual society that considers Malay to be the national language but uses mixed languages in daily life. Bahasa Rojak (Bukhari et al., 2015) is one of the code-mixing examples that combines Malay and English into ∗Corresponding author. 1Both BRCC and SentiBahasaRojak are available at https://data.depositar.io/dataset/brcc_ and_sentibahasarojak a certain level of language structure. Bahasa Ro- jak or Malaysian English (Vollmann and Wooi, 2019) is often mixed with Chinese because the ethnic Chinese population in Malaysia is quite large. Bakar and Mazzalan (2018) show that many users in Malaysia use Bahasa Rojak on Facebook. We can find many code-mixing combinations like Malay-English, Hindi-English, Spanish-English, and others on social media. People not only ex- press their feelings on social media, but also ex- change information such as the latest news from their hometowns or countries, and financial topics are also popular. Therefore, natural language pro- cessing (NLP) studies are increasingly focused on code-mixing (Thara and Poornachandran, 2018). One of the most advanced trends in natural lan- guage processing is to prepare a large unlabeled corpus to pre-train a language model for represent- ing the input text. The source of this unlabeled cor- pus is usually Wikipedia (Qiu et al., 2020). How- ever, Wikipedia does not have any Bahasa Rojak pages, which hinders the training of a pre-trained model that can represent Bahasa Rojak input texts. As a …

Content truncated to 500 words