Neural Machine Translation for Sinhala-English Code-Mixed Text

Proceedings of Recent Advances in Natural Language Processing, pages 718–726 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_082 718 Neural Machine Translation for Sinhala-English Code-Mixed Text Archchana Kugathasan Department Of Computational Mathematics University Of Moratuwa Katubedda archchanakugathasan@gmail.com Sagara Sumathipala Department Of Computational Mathematics University Of Moratuwa Katubedda sagaras@uom.lk Abstract Code-mixing has become a moving method of communication among multilingual speakers. Most of the social media con- tent of the multilingual societies are writ- ten in code-mixed text. However, most of the current translation systems neglect to convert code-mixed texts to a stan- dard language. Most of the user writ- ten code-mixed content in social media re- mains unprocessed due to the unavailabil- ity of linguistic resource such as parallel corpus. This paper proposes a Neural Ma- chine Translation(NMT) model to trans- late the Sinhala-English code-mixed text to the Sinhala language. Due to the lim- ited resources available for Sinhala-English code-mixed(SECM) text, a parallel corpus is created with SECM sentences and Sin- hala sentences. Srilankan social media sites contain SECM texts more frequently than the standard languages. The model proposed for code-mixed text translation in this study is a combination of Encoder- Decoder framework with LSTM units and Teachers Forcing Algorithm. The trans- lated sentences from the model are evalu- ated using BLEU(Bilingual Evaluation Un- derstudy) metric. Our model achieved a re- markable BLEU score for the translation. 1 Introduction Before 1990, translation was considered a diffi- cult task due to many reasons such as ambigu- ity, translation mismatch, co-reference, trans- lation divergence and development of language over time(Sreelekha et al., 2016) but Machine Translation(MT) since 1990 has been a vast and successful research area in natural lan- guage processing. Machine translation has been given importance in the research field be- cause it is used to translate texts for military authorities to track enemies, foreign business collaborations, marketing, etc.(Kalchbrenner and Blunsom, 2013). Expressing the thoughts of the personal in- terests, daily life etc., of a person in social me- dia networks has become a trending activity among people. Texts extracted from social me- dia lead to measure the social dynamics of sev- eral societies(Arguello et al., 2008). Content- based search engines, personalized advertise- ments, recommendation systems, etc. use the user-generated content from social media to increase their value and to provide more ac- curate results to the users(Sippel and Brodt, 2008). Processing a content in standard lan- guage(without code-mixing) is considered as an easy task, where the text with code-mixing has been considered as a road block to extract the needed information. Code-mixing is con- sidered as an invention of bilingualism and multilingualism. The capability of speaking in two languages, is called bilingualism and more than two languages is called multilin- gualism. Most of the Srilankans are bilingual. The user-generated content such as posts, com- ments,reviews etc., in Srilankan social media are mostly in SECM text. The main focus of this study is to translate the SECM text to Sinhala language. To understand the necessity of processing SECM text, a survey study was conducted among …

Content truncated to 500 words