Pattern Matching Refinements to Dictionary-Based Code-Switching Point Detection

Copyright 2012 by Nathaniel Oco and Rachel Edita Roxas 26th Pacific Asia Conference on Language,Information and Computation pages 229–236 Pattern Matching Refinements to Dictionary-Based Code-Switching Point Detection Nathaniel Oco Rachel Edita Roxas De La Salle University De La Salle University 2401 Taft Avenue Malate, Manila City 2401 Taft Avenue Malate, Manila City 1004 Metro Manila, Philippines 1004 Metro Manila, Philippines nathan.oco@delasalle.ph rachel.roxas@delasalle.ph Abstract This study presents the development and evaluation of pattern matching refinements (PMRs) to automatic code switching point (CSP) detection. With all PMRs, evaluation showed an accuracy of 94.51%. This is an improvement to reported accuracy rates of dictionary-based approaches, which are in the range of 75.22%-76.26% (Yeong and Tan, 2010). In our experiments, a 100- sentence Tagalog-English corpus was used as test bed. Analyses showed that the dictionary-based approach using part-of- speech checking yielded an accuracy of 79.76% only, and two notable linguistic phenomena, (1) intra-word code-switching and (2) common words, were shown to have caused the low accuracy. The devised PMRs, namely: (1) common word exclusion, (2) common word identification, and (3) common n-gram pruning address this and showed improved accuracy. The work can be extended using audio files and machine learning with larger language resources. 1 Introduction Code-switching (CS) is “the use of two or more linguistic varieties in the same interaction or conversation” (Myers-Scotton and Ury, 1977). It is often prevalent in communities where there is language contact. According to linguistic studies (Bautista, 1991; Bautista, 2004; Borlongan, 2009), code-switching reasons are mainly driven by proficiency or deficiency in the languages involved. Proficiency-driven code-switching takes place when a person is competent with the two languages and can easily switch from one to the other “for maximum efficiency or effect”. On the other hand, deficiency-driven code-switching takes place when people are forced to code-switch to one language because they are “not competent in the use of the other language”. Oral communication in both languages can be enhanced by the detection of code-switching points (CSPs). To detect CSPs, we developed a dictionary-based approach using a rule-based engine (Naber, 2003), and we also developed pattern matching refinements (PMRs) to improve accuracy. As testbed, this study focuses on Tagalog- English code-switching, which can be classified into (1) intra-sentential and (2) intra-word code- switching. Intra-sentential CS is the switching between Tagalog and English words and clauses, while intra-word CS is the use of English root words with Tagalog affixes and morphological rules. An example of intra-sentential CS is “Unless let us say may mga bisita siya” (translated as: Unless let us say he/she has visitors) and an example of intra-word CS is “nagdadrive” (incompeleted aspect of the English verb “drive”). The system developed can effectively be used to detect intra-sentential (Tagalog to English and English to Tagalog) and intra-word CSPs. 229 This paper is organized as follows: related works in section 2, CSP detection in section 3, pattern matching refinements in section 4, testing and discussion in section 5, and conclusion in section 6. 2 Related Works In the field …

Content truncated to 500 words