Effectiveness of Character Language Model for Vietnamese Named Entity Recognition

Effectiveness of Character Language Model for Vietnamese Named Entity Recognition Xuan-Dung Doan, Trung-Thanh Dang Viettel Cyberspace Center {dungdx4,thanhdt13}@viettel.com.vn Le-Minh Nguyen Japan Advanced Institute of Science and Technology nguyenml@jaist.ac.jp Abstract Recently, many studies indicate that charac- ter language model can capture syntactic- semantic word features, resulting in state-of- the-art performance in typical NLP sequence labeling tasks. This paper shows the effective- ness of character language model for Viet- namese Named Entity Recognition by com- paring several methods. We evaluate the pro- posed model on the VLSP 2016 dataset and our own VTNER dataset. Experimental results show that our model is the current state-of-the- art end-to-end obtains for the task. 1 Introduction Named-entity recognition (NER) is the task of au- tomatically identifying and classifying elements of the document into several categories, such as orga- nization, person, location, currency, time, etc. NER is used in data mining systems, text summarization, question answering, translation, etc. Most methods for NER are based on machine learning. For example, given the following sentence as in- put: Đại học Bách khoa Hà Nội nằm trên đường Đại Cồ Việt. (The Hanoi Univer- sity of Science and Technology is on Dai Co Viet street.) The output is: [Đại học Bách khoa Hà Nội]ORGANIZATION nằm trên [đường Đại Cồ Việt]LOCATION. Many methods for NER are based on supervised learning. In particular, Conditional Random Field - CRF (Lafferty et al., 2001; Sutton and McCal- lum, 2006) and Long Short Term Memory - LSTM (Hochreiter and Schmidhuber, 1997) are the most popular methods. (Le, 2016) combined regular expressions over to- kens and a bidirectional inference method in a se- quence labeling model. (Pham and Le, 2017) com- bined Bi-LSTM, CNN and CRF that achieved the same performance with (Le, 2016). This system is the end-to-end architecture that required only word embeddings. After that, (Pham and Le, 2017) system surpassed both (Le, 2016) and end-to-end (Pham and Le, 2017) systems by using Bi-LSTM with automatically syntactic to present a state-of-the-art named entity recognition system for the Vietnamese language. Minh (2018) showed the effectiveness of rich features on CRF methods by using default fea- tures for CRF with POS and chunking tags and achieved best results on F1 score. Recently, Contextualized word embeddings (Pe- ters et al., 2017; Peters et al., 2018) capture word sematics in context to address the polysemous and context-dependent nature of words. They report new state-of-the-art results for NER but this approach require a larger model, external corpus and time- consuming training. (Liu et al., 2018) proposed a sequence labeling framework, LM-LSTM-CRF, and (Akbik et al., 2018) suggested Contextual String Embeddings, both of which achieved state-of-the- art results in English datasets. Their model leverages both word-level and character-level knowledge. Thus, we wanted to implement the methods: CRF, PACLIC 32 157 32nd Pacific Asia Conference on Language, Information and Computation Hong Kong, 1-3 December 2018 Copyright 2018 by the authors LSTM-CRF and checked the effectiveness of hand- crafted features on the models. Besides, we used the character language model that (Liu …

Content truncated to 500 words