Construction of a VerbNet style lexicon for Vietnamese

Contextual Characters with Segmentation Representation for Named Entity Recognition in Chinese BLOUIN Baptiste Aix-Marseille University, IrAsia Aix-Marseille University, LIS ENP-China baptiste.blouin@lis-lab.fr MAGISTRY Pierre Aix-Marseille University, IrAsia ENP-China pierre@magistry.fr Abstract Named Entity Recognition (NER) is a typi- cal sequence labeling task. It remains chal- lenging for Chinese, partly because of the lack of clear typographic word boundaries. Deci- sions have to be made regarding the choice of basic units which constitute the sequence to be labeled, and their vectorized represen- tation. Recent approaches have shown that character-based models lack the information about larger units (words) which is useful for NER, while word-based models may suffer from the propagation of word segmentation er- rors and a higher rate of Out-of-Vocabulary (OOV) tokens. In this paper, we propose a new representation of sinograms (Chinese characters) enriched with word boundary in- formation, for which different types of em- beddings can be built. Experiments show that our solution outperforms other state-of-the-art models. We also took great care to propose a fully retrainable pipeline, which is available at https://github.com/enp-china/CCSR-NER . It does not rely on pretrained models and can be trained in few days on common hardware. 1 Introduction The present work explores the task of Named Entity Recognition (NER) in Mandarin Chinese, specifi- cally for cases when relying on large pre-trained models is not an option. This can occur when one has to process domain specific data, or in our case1, historical texts where language is quite dif- ferent from the language of the corpora used to pretrain publicly available models, especially words and characters embeddings. The models we propose can be trained in a reasonable time (days) from a rel- atively small amount of raw data (few hundred mil- lions of characters) on affordable hardware (such as a single GTX 1080 ti). 1ENP China, https://www.enpchina.eu/ (ERC No 788476) Chinese script does not provide a clear and fre- quent typographic marker for word boundaries. As a result, when addressing the case of Chinese(s) language(s) in NER, we have to face the issue of word segmentation. Recent models proposed in the literature can be divided into character-based, word-based or hybrid models, but every work had to take a stance regarding Chinese Word Segmenta- tion (CWS). The importance and methods for CWS have a long history in Chinese NLP, a recent work Li et al. (2019) makes the strong claim that the neu- ral era of NLP is turning CWS into an irrelevant or even harmful step in a pipeline. However Li et al. (2019) did not provide experimental results on the NER task and our own experiments presented in this paper tend to show that CWS can be either harmful or beneficial, depending on how much care is given to consistency in segmentation and to the way word embeddings are built and used. Our main findings are that off-the-shelf embeddings for Man- darin Chinese must be used carefully, but it is pos- sible to improve on the state-of-the-art by retrain- ing everything from raw and labeled …

Content truncated to 500 words