Using Stanford Part-of-Speech Tagger for the Morphologically-rich Filipino Language

31st Pacific Asia Conference on Language, Information and Computation (PACLIC 31), pages 81–88 Cebu City, Philippines, November 16-18, 2017 Copyright c⃝2017 Matthew Phillip Go and Nicco Nocon Using Stanford Part-of-Speech Tagger for the Morphologically-rich Filipino Language Matthew Phillip V. Go De la Salle University Manila, Philippines matthew.go123@gmail.com Nicco Nocon De la Salle University Manila, Philippines noconoccin@gmail.com Abstract This research focuses on the implementation of a Maximum Entropy-based Part-of-Speech (POS) tagger for Filipino. It uses the Stan- ford POS tagger – a trainable POS tagger that has been trained on English, Chinese, Arabic, and other languages and producing one of the highest results in each language. The tagger was trained for Filipino using a 406k token corpus and considering unique Filipino lin- guistic phenomena such as high morphology and intra-sentential code-switches. The Fil- ipino POS tagger resulted to 96.15% tagging accuracy which currently presents the highest accuracy and with a large lead among existing POS taggers for Filipino. 1 Introduction A Part-of-Speech (POS) tagger is a software that classifies words into its word classes or lexical cate- gories (Bird et al., 2009). POS tags and taggers have proven its importance in Natural Language Process- ing (NLP) when used in advanced NLP researches such as grammar checkers (Go and Borra, 2016), information extraction (Surdeanu et al., 2011), and word-sense disambiguation (Chen et al., 2009). In a pipeline architecture of an advanced research such as an information extraction system, POS taggers are usually found in the first section producing POS tags or tag sequences. These POS tags may be used as basic features or to produce more advanced features such as syntactic structures using a con- stituency parser and dependencies between words using a dependency parser (Surdeanu et al., 2011; Chen and Manning, 2014). Despite being a fundamental NLP tool towards advanced NLP researches, there seems to be few re- searches made towards the development of a high- performing POS tagger for Filipino, the national lan- guage of the Philippines – a Southeast Asian country with a population of 101 million people1. The following are the list of POS taggers devel- oped for Tagalog, the dialect from where Filipino was based on: TPOST (Rabo and Cheng, 2006), MBPOST (Raga and Trogo, 2006), PTPOST4.1 (Go, 2006), Tag-Alog (Fontanilla and Wu, 2006), and SVPOST (Reyes et al., 2011); Adding to the list is the recently published POS tagger designed for Filipino named SMTPOST (Nocon and Borra, 2016). The key difference between Tagalog and Filipino is the presence of accepted English words such as ‘cellphone’, ‘laptop’, ‘professor’, ‘polo shirt’ as part of the Filipino language leading to nonce borrowings (single word code switching) and even intra-word code switching such as nag-conduct ‘conducted’ (added prefix nag-) and tinetext ‘texting (someone)’ (added infix -in-). Looking into the design of the taggers, TPOST and MBPOST are closely similar because both sys- tems utilize a lexicon list, surrounding words, capi- talization, and affix features using a stemmer; where tagging rules are extracted from the training to be used during testing. …

Content truncated to 500 words