Improving Low-Resource Named Entity Recognition using Joint Sentence and Token Labeling

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5898–5905 July 5 - 10, 2020. c⃝2020 Association for Computational Linguistics 5898 Improving Low-Resource Named Entity Recognition using Joint Sentence and Token Labeling Canasai Kruengkrai Thien Hai Nguyen Sharifah Mahani Aljunied Lidong Bing DAMO Academy, Alibaba Group canasai@gmail.com, {thienhai.nguyen,mahani.aljunied,l.bing}@alibaba-inc.com Abstract Exploiting sentence-level labels, which are easy to obtain, is one of the plausible methods to improve low-resource named entity recogni- tion (NER), where token-level labels are costly to annotate. Current models for jointly learn- ing sentence and token labeling are limited to binary classification. We present a joint model that supports multi-class classification and introduce a simple variant of self-attention that allows the model to learn scaling factors. Our model produces 3.78%, 4.20%, 2.08% improvements in F1 over the BiLSTM-CRF baseline on e-commerce product titles in three different low-resource languages: Vietnamese, Thai, and Indonesian, respectively. 1 Introduction Neural named entity recognition (NER) has be- come a mainstream approach due to its superior performance (Huang et al., 2015; Lample et al., 2016; Ma and Hovy, 2016; Chiu and Nichols, 2016; Akbik et al., 2018). However, neural NER typi- cally requires a large amount of manually labeled training data, which are not always available in low-resource languages. Training neural NER with limited labeled data can be very challenging. In this paper, we consider bridging multi-task learn- ing (MTL) (Caruana, 1993; Ruder, 2017) and pre- training (Peters et al., 2018; Devlin et al., 2019) to leverage training signals of an auxiliary task that has a sufficiently large number of labeled data. Researchers have investigated a wide variety of auxiliary tasks and resources to boost the per- formance of neural NER, e.g., training coarse- grained NER (Aguilar et al., 2017), fine-tuning bilingual word embeddings (Wang et al., 2017), applying language models (Rei, 2017), integrating part-of-speech (POS) tagging (Lin et al., 2018), using cross-lingual knowledge (Feng et al., 2018), and learning paraphrases (Watanabe et al., 2019). Category: HEALTH_BEAUTY Title: COMBO Gôm xịt tóc Tigi Bed Head Label: O B-PRODUCT I-PRODUCT E-PRODUCT B-BRAND I-BRAND E-BRAND Translation: combo hairspray Tigi Bed Head ‘‘… Tigi Bed Head hairspray combo …’’ Category: ELECTRONICS Title: Ốp lưng silicon dẻo Hàn Quốc Label: B-PRODUCT E-PRODUCT S-MATERIAL S-PATTERN O O Translation: case silicon flexible Korea ‘‘… Korean flexible silicon case …’’ Figure 1: Examples of product titles with NER annota- tion in Vietnamese. Product categories are provided by sellers and can be used as sentence-level labels. While most of the previous studies have exploited token-level information from auxiliary tasks, a few of them have tried to use sentence-level informa- tion (Rei and Søgaard, 2018; Devlin et al., 2019). Our work is closely related to the joint labeling framework in Rei and Søgaard (2019). However, they only focused on binary classification, while we attempt to handle multi-class classification on both sentence and token levels. In this work, we focus on improving low- resource NER by exploiting large data, only having sentence-level labels. Figure 1 shows examples of product titles on …

Content truncated to 500 words