IndoCollex: A Testbed for Morphological Transformation of Indonesian Colloquial Words

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3170–3183 August 1–6, 2021. ©2021 Association for Computational Linguistics 3170 IndoCollex: A Testbed for Morphological Transformation of Indonesian Colloquial Words Haryo Akbarianto Wibowo∗ Made Nindyatama Nityasya∗ Afra Feyza Aky¨urek⋆ Suci Fitriany∗ Alham Fikri Aji∗ Radityo Eko Prasojo∗• Derry Tanti Wijaya⋆ ∗ Kata.ai Research Team, Jakarta, Indonesia ⋆ Department of Computer Science, Boston University • Faculty of Computer Science, Universitas Indonesia {haryo,made,suci,aji,ridho}@kata.ai Abstract Indonesian language is heavily riddled with colloquialism whether in written or spoken forms. In this paper, we identify a class of Indonesian colloquial words that have un- dergone morphological transformations from their standard forms, categorize their word for- mations, and propose a benchmark dataset of Indonesian Colloquial Lexicons (IndoCollex) consisting of informal words on Twitter ex- pertly annotated with their standard forms and their word formation types/tags. We evalu- ate several models for character-level trans- duction to perform morphological word nor- malization on this testbed to understand their failure cases and provide baselines for future work. As IndoCollex catalogues word forma- tion phenomena that are also present in the non-standard text of other languages, it can also provide an attractive testbed for methods tailored for cross-lingual word normalization and non-standard word formation. 1 Introduction Indonesian language is one of the most widely spo- ken languages in the world with around 200 million speakers. Despite its large number of speakers, in terms of NLP resources, Indonesian language is not very well represented (Joshi et al., 2020). Most of its data are in the form of unlabeled web and user generated contents in online platforms such as social media, which are noisy and riddled with colloquialism which poses difficulties for NLP sys- tems (Baldwin et al., 2013a; Eisenstein, 2013a). Traditionally, the majority of Indonesian col- loquial or informal lexicons are borrowed words from foreign or local dialect words, and sometimes with phonetic and lexical modifications.1 Increas- ingly however, Indonesian colloquial words are 1For example, gue, a common informal form of aku (‘I’, ‘me’), is a word that originates from the Betawi dialect. more commonly a morphological transformation2 of their standard counterparts.3 Despite these evolv- ing lexicons, existing research on Indonesian word normalization has largely (1) relied on creating static informal dictionaries (Le et al., 2016), render- ing normalization of unseen words impossible, and (2) for the specific task of sentiment analysis (Le et al., 2016) or machine translation (Guntara et al., 2020), with no direct implication to word normal- ization in general. Given the obvious utility of cre- ating NLP systems that can normalize Indonesian informal data, we believe that the bottleneck is that there is no standard open testbed for researchers and developers of such system to test the effective- ness of their models to these colloquial words. In this paper, we introduce IndoCollex, a new, re- alistic dataset aimed at testing normalization mod- els to these phenomena. IndoCollex is a profession- ally annotated dataset, where each informal word is paired with its standard form and expertly an- notated …

Content truncated to 500 words