Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3913–3921 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3913 Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts Oanh Thi Tran∗,†, Tu Minh Pham∗, Vu Hoang Dang∗, Bang Ba Xuan Nguyen∗ ∗FPT Technology Research Institute - FPT University 82 Duy Tan, Cau Giay, Hanoi, Vietnam (oanhtt12, tupm2, vudh5, bangnbx)@fpt.com.vn †International School, Vietnam National University, Hanoi 144 Xuan Thuy, Cau Giay, Hanoi, Vietnam oanhtt@isvnu.vn Abstract This paper introduces a large-scale human-labeled dataset for the Vietnamese POS tagging task on conversational texts. To this end, we propose a new tagging scheme (with 36 POS tags) consisting of exclusive tags for special phenomena of conversational words, develop the annotation guideline and manually annotate 16.310K sentences using this guideline. Based on this corpus, a series of state-of-the-art tagging methods has been conducted to estimate their performances. Experimental results showed that the Conditional Random Fields model using both automatically learnt features from deep neural networks and handcrafted features yielded the best performance. This model achieved 93.36% in the accuracy score which is 1.6% and 2.7% higher than the model using either handcrafted features or automatically-learnt features, respectively. This result is also a little bit higher than the model of fine-tuning BERT by 0.94% in the accuracy score. The performance measured on each POS tag is also very high with >90% in the F1 score for 20 POS tags and >80% in the F1 score for 11 POS tags. This work provides the public dataset and preliminary results for follow-up research on this interesting direction. Keywords: Vietnamese POS tagging, conversational texts, CRF, neural networks 1. Introduction POS tagging is one of the most critical tasks in Natural Language Processing (NLP) and has drawn much attention among its research community. Its performance usually greatly affect many NLP downstream tasks such as question answering, machine translation, named entity recognition, sentiment analysis, etc. So far, most POS tagging work has been dedicated to formal (or standardized) texts such as e-newspapers, official documents, and stories. Along with the development of social media, the amount and types of data available from Twitter, Facebook, blogs, and chat-chit platforms are increasing exponentially. Unfor- tunately, most current POS taggers typically perform poorly on these kinds of social data1. The main reason is that these POS taggers were trained on standardized texts which are from a quite different domain (Gimpel et al., 2011). Com- pared with these formal texts, informal texts usually contain many informal inputs, such as acronyms (e.g also known as –> aka), abbreviation (e.g technology –> tech), out-of- vocabulary words (alooooo –> alo), no accent markers, etc. Recognizing its POS tags, hence is a challenging task esp. for poor-resource languages like Vietnamese. In fact, most work about POS tagging on social media texts was dedicated to popular languages such as English (Gimpel et al., 2011), Chinese (Wang et al., 2019), Spanish (Meftah and Semmar, 2018), etc. Meanwhile, very little work was …

Content truncated to 500 words