A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing

A neural joint model for Vietnamese word segmentation, POS tagging and dependency parsing Dat Quoc Nguyen1,2 1The University of Melbourne, Australia dqnguyen@unimelb.edu.au 2VinAI Research, Hanoi, Vietnam v.datnq9@vinai.io Abstract We propose the first multi-task learning model for joint Vietnamese word segmentation, part- of-speech (POS) tagging and dependency parsing. In particular, our model extends the BIST graph-based dependency parser (Kiper- wasser and Goldberg, 2016) with BiLSTM- CRF-based neural layers (Huang et al., 2015) for word segmentation and POS tagging. On Vietnamese benchmark datasets, experimental results show that our joint model obtains state- of-the-art or competitive performances. 1 Introduction Dependency parsing (K¨ubler et al., 2009) is ex- tremely useful in many downstream applications such as relation extraction (Bunescu and Mooney, 2005) and machine translation (Galley and Man- ning, 2009). POS tags are essential features used in dependency parsing. In real-world parsing, most parsers are used in a pipeline process with a precursor POS tagging model for producing pre- dicted POS tags. In English where white space is a strong word boundary indicator, POS tagging is considered to be the first important step towards dependency parsing (Ballesteros et al., 2015). Unlike English, for Vietnamese NLP, word seg- mentation is considered to be the key first step. This is because when written, white space is used in Vietnamese to separate syllables that consti- tute words, in addition to marking word bound- aries (Nguyen et al., 2009). For example, a 4- syllable written text “Tˆoi l`a sinh viˆen” (I am stu- dent) forms 3 words “TˆoiI l`aam sinh viˆenstudent”.1 When parsing real-world Vietnamese text where gold word segmentation is not available, a pipeline process is defined that starts with a word seg- menter to segment the text. The segmented text 1About 85% of Vietnamese word types are composed of at least two syllables and 80%+ of syllable types are words by themselves (Thang et al., 2008). For Vietnamese word seg- mentation, white space is only used to separate word tokens while underscore is used to separate syllables inside a word. LSTM LSTM LSTM I B O Tôi là sinh viên Initial word-boundary tag embedding Syllable vector representation Word segmentation O CRF BiLSTMWS Syllable embedding LSTM LSTM LSTM LSTM PRON BiLSTMDEP LSTM POS tagging Dependency parsing VERB LSTM CRF BiLSTMPOS Word vector representation Tôi là sinh_viên Word embedding LSTM NOUN sub vmod softmax FFNN softmax FFNN ID Form POS Head DepRel 1 Tˆoi I PRON 2 sub 2 l`a am VERB 0 root 3 sinh viˆen student NOUN 2 vmod Figure 1: Illustration of our joint model. Linear trans- formations are not shown for simplification. (e.g. “Tˆoi l`a sinh viˆen”) is provided as the input to the POS tagger, which automatically generates POS-annotated text (e.g. “Tˆoi/PRON l`a/VERB sinh viˆen/NOUN”) which is in turn fed to the parser. See Figure 1 for the final parsing output. However, Vietnamese word segmenters and POS taggers have a non-trivial error rate, thus leading to error propagation. A solution to these problems is to develop models for jointly learn- ing word segmentation, …

Content truncated to 500 words