Low-Resource Name Tagging Learned with Weakly Labeled Data

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 261–270, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 261 Low-Resource Name Tagging Learned with Weakly Labeled Data Yixin Cao1 Zikun Hu1 Tat-Seng Chua1 Zhiyuan Liu2 Heng Ji3 1School of Computing, National University of Singapore, Singapore 2Department of CST, Tsinghua University, Beijing, China 3Department of CS, University of Illinois Urbana-Champaign, U.S.A. caoyixin2011@gmail.com,zikunhu@u.nus.edu,dcscts@nus.edu.sg liuzy@tsinghua.edu.cn,hengji@illinois.edu Abstract Name tagging in low-resource languages or domains suffers from inadequate training data. Existing work heavily relies on additional in- formation, while leaving those noisy annota- tions unexplored that extensively exist on the web. In this paper, we propose a novel neu- ral model for name tagging solely based on weakly labeled (WL) data, so that it can be applied in any low-resource settings. To take the best advantage of all WL sentences, we split them into high-quality and noisy portions for two modules, respectively: (1) a classifi- cation module focusing on the large portion of noisy data can efficiently and robustly pre- train the tag classifier by capturing textual con- text semantics; and (2) a costly sequence la- beling module focusing on high-quality data utilizes Partial-CRFs with non-entity sampling to achieve global optimum. Two modules are combined via shared parameters. Extensive experiments involving five low-resource lan- guages and fine-grained food domain demon- strate our superior performance (6% and 7.8% F1 gains on average) as well as efficiency1. 1 Introduction Name tagging2 is the task of identifying the boundaries of entity mentions in texts and classi- fying them into the pre-defined entity types (e.g., person). It serves as a fundamental role as pro- viding the essential inputs for many IE tasks, such as Entity Linking (Cao et al., 2018a) and Relation Extraction (Lin et al., 2017). Many recent methods utilize a neural net- work (NN) with Conditional Random Fields (CRFs) (Lafferty et al., 2001) by treating name tagging as a sequence labeling problem (Lample 1Our project can be found in https://github.com/ zig-kwin-hu/Low-Resource-Name-Tagging. 2Someone may call it Named Entity Recognition (NER). … Barangay Ginebra and Formula Shell forming a rivalry … Barangay Ginebra and Formula Shell forming a rivalry ShaquilleO’Neal … Barangay Ginebra and Formula Shell forming a rivalry B-ORG O O O O B-LOC I-ORG Formula shell won game one in Philippines … Weakly Labelled, extensively exists on the web Fully Labelled, expensive to obtain s1 <latexit sha1_base64="zD0+iHg15Vq/defZn4JzGP6vt+A=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV RIRdFl02VF+4BaSjKd1tC8yEyUgR/wK1+mvgH+hfeGaegFtEJSc6ce8+Zuf6aRgI6TivBWthcWl5pbhaWlvf2Nwqb+0RJnjDdZEiZx/cED4OYN2UgQ95JM+5Ffsjb/vhcxdu3PBNBEl/JScp7kTeKg2HAPEnUpei7/ XLFqTp62fPANaACsxpJ+QXGCABQ4IHDEk4RAeBD1duHCQEtfDlLiMUKDjHPcokTanLE4ZHrFj+o5o1zVsTHvlKbSa0SkhvRkpbRyQJqG8jLA6zdbxXDsr9jfvqfZUd5vQ3zdeEbESN8T+pZtl/lenapEY4lTXEFBNqWZUd cy45Lor6ub2l6okOaTEKTygeEaYaeWsz7bWCF276q2n4286U7Fqz0xujnd1Sxqw+3Oc86B1VHWdqntxXKmdmVEXsYd9HNI8T1BDHQ0yXuERzh2apbsZVbd5+pVsFodvFtWQ8fAiGQHA=</latexit> <latexit sha1_base64="zD0+iHg15Vq/defZn4JzGP6vt+A=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV RIRdFl02VF+4BaSjKd1tC8yEyUgR/wK1+mvgH+hfeGaegFtEJSc6ce8+Zuf6aRgI6TivBWthcWl5pbhaWlvf2Nwqb+0RJnjDdZEiZx/cED4OYN2UgQ95JM+5Ffsjb/vhcxdu3PBNBEl/JScp7kTeKg2HAPEnUpei7/ XLFqTp62fPANaACsxpJ+QXGCABQ4IHDEk4RAeBD1duHCQEtfDlLiMUKDjHPcokTanLE4ZHrFj+o5o1zVsTHvlKbSa0SkhvRkpbRyQJqG8jLA6zdbxXDsr9jfvqfZUd5vQ3zdeEbESN8T+pZtl/lenapEY4lTXEFBNqWZUd cy45Lor6ub2l6okOaTEKTygeEaYaeWsz7bWCF276q2n4286U7Fqz0xujnd1Sxqw+3Oc86B1VHWdqntxXKmdmVEXsYd9HNI8T1BDHQ0yXuERzh2apbsZVbd5+pVsFodvFtWQ8fAiGQHA=</latexit> <latexit sha1_base64="zD0+iHg15Vq/defZn4JzGP6vt+A=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV RIRdFl02VF+4BaSjKd1tC8yEyUgR/wK1+mvgH+hfeGaegFtEJSc6ce8+Zuf6aRgI6TivBWthcWl5pbhaWlvf2Nwqb+0RJnjDdZEiZx/cED4OYN2UgQ95JM+5Ffsjb/vhcxdu3PBNBEl/JScp7kTeKg2HAPEnUpei7/ XLFqTp62fPANaACsxpJ+QXGCABQ4IHDEk4RAeBD1duHCQEtfDlLiMUKDjHPcokTanLE4ZHrFj+o5o1zVsTHvlKbSa0SkhvRkpbRyQJqG8jLA6zdbxXDsr9jfvqfZUd5vQ3zdeEbESN8T+pZtl/lenapEY4lTXEFBNqWZUd cy45Lor6ub2l6okOaTEKTygeEaYaeWsz7bWCF276q2n4286U7Fqz0xujnd1Sxqw+3Oc86B1VHWdqntxXKmdmVEXsYd9HNI8T1BDHQ0yXuERzh2apbsZVbd5+pVsFodvFtWQ8fAiGQHA=</latexit> <latexit sha1_base64="zD0+iHg15Vq/defZn4JzGP6vt+A=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV RIRdFl02VF+4BaSjKd1tC8yEyUgR/wK1+mvgH+hfeGaegFtEJSc6ce8+Zuf6aRgI6TivBWthcWl5pbhaWlvf2Nwqb+0RJnjDdZEiZx/cED4OYN2UgQ95JM+5Ffsjb/vhcxdu3PBNBEl/JScp7kTeKg2HAPEnUpei7/ XLFqTp62fPANaACsxpJ+QXGCABQ4IHDEk4RAeBD1duHCQEtfDlLiMUKDjHPcokTanLE4ZHrFj+o5o1zVsTHvlKbSa0SkhvRkpbRyQJqG8jLA6zdbxXDsr9jfvqfZUd5vQ3zdeEbESN8T+pZtl/lenapEY4lTXEFBNqWZUd cy45Lor6ub2l6okOaTEKTygeEaYaeWsz7bWCF276q2n4286U7Fqz0xujnd1Sxqw+3Oc86B1VHWdqntxXKmdmVEXsYd9HNI8T1BDHQ0yXuERzh2apbsZVbd5+pVsFodvFtWQ8fAiGQHA=</latexit> s2 <latexit sha1_base64="9HpV6YLUpskfSajXTx2xU6cogv8=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV ZIi6LopsuKthVqKUk6rUPzYmailCL4A27108Q/0L/wzpiCWkQnJDlz7j1n5t7rpyGXynFeC9bC4tLySnG1tLa+sblV3t5pyQTAWsFSZiIK9+TLOQxaymuQnaVCuZFfsg6/vhMxzu3TEiexJdqkrJe5I1iPuSBp4i6kP1av 1xqo5Z9jxwc1BvpJ+QXGCBgAwRGIowiE8SHq6cOEgJa6HKXGCEDdxhnuUSJtRFqMj9gxfUe06+ZsTHvtKY06oFNCegUpbRyQJqE8QVifZpt4Zpw1+5v31Hjqu03o7+deEbEKN8T+pZtl/lena1EY4sTUwKm1DC6u iB3yUxX9M3tL1UpckiJ03hAcUE4MpZn2jkaZ23VvPxN9Mpmb1PshzM7zrW9KA3Z/jnAftWtV1qu75UaV+mo+6iD3s45DmeYw6GmiRd4jPOIJz1bDiq3MuvtMtQq5ZhflvXwAQSBkB0=</latexit> <latexit sha1_base64="9HpV6YLUpskfSajXTx2xU6cogv8=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV ZIi6LopsuKthVqKUk6rUPzYmailCL4A27108Q/0L/wzpiCWkQnJDlz7j1n5t7rpyGXynFeC9bC4tLySnG1tLa+sblV3t5pyQTAWsFSZiIK9+TLOQxaymuQnaVCuZFfsg6/vhMxzu3TEiexJdqkrJe5I1iPuSBp4i6kP1av 1xqo5Z9jxwc1BvpJ+QXGCBgAwRGIowiE8SHq6cOEgJa6HKXGCEDdxhnuUSJtRFqMj9gxfUe06+ZsTHvtKY06oFNCegUpbRyQJqE8QVifZpt4Zpw1+5v31Hjqu03o7+deEbEKN8T+pZtl/lena1EY4sTUwKm1DC6u iB3yUxX9M3tL1UpckiJ03hAcUE4MpZn2jkaZ23VvPxN9Mpmb1PshzM7zrW9KA3Z/jnAftWtV1qu75UaV+mo+6iD3s45DmeYw6GmiRd4jPOIJz1bDiq3MuvtMtQq5ZhflvXwAQSBkB0=</latexit> <latexit sha1_base64="9HpV6YLUpskfSajXTx2xU6cogv8=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV ZIi6LopsuKthVqKUk6rUPzYmailCL4A27108Q/0L/wzpiCWkQnJDlz7j1n5t7rpyGXynFeC9bC4tLySnG1tLa+sblV3t5pyQTAWsFSZiIK9+TLOQxaymuQnaVCuZFfsg6/vhMxzu3TEiexJdqkrJe5I1iPuSBp4i6kP1av 1xqo5Z9jxwc1BvpJ+QXGCBgAwRGIowiE8SHq6cOEgJa6HKXGCEDdxhnuUSJtRFqMj9gxfUe06+ZsTHvtKY06oFNCegUpbRyQJqE8QVifZpt4Zpw1+5v31Hjqu03o7+deEbEKN8T+pZtl/lena1EY4sTUwKm1DC6u iB3yUxX9M3tL1UpckiJ03hAcUE4MpZn2jkaZ23VvPxN9Mpmb1PshzM7zrW9KA3Z/jnAftWtV1qu75UaV+mo+6iD3s45DmeYw6GmiRd4jPOIJz1bDiq3MuvtMtQq5ZhflvXwAQSBkB0=</latexit> <latexit sha1_base64="9HpV6YLUpskfSajXTx2xU6cogv8=">ACxnicjVHLSsNAFD2Nr1pfVZdugkVwV ZIi6LopsuKthVqKUk6rUPzYmailCL4A27108Q/0L/wzpiCWkQnJDlz7j1n5t7rpyGXynFeC9bC4tLySnG1tLa+sblV3t5pyQTAWsFSZiIK9+TLOQxaymuQnaVCuZFfsg6/vhMxzu3TEiexJdqkrJe5I1iPuSBp4i6kP1av 1xqo5Z9jxwc1BvpJ+QXGCBgAwRGIowiE8SHq6cOEgJa6HKXGCEDdxhnuUSJtRFqMj9gxfUe06+ZsTHvtKY06oFNCegUpbRyQJqE8QVifZpt4Zpw1+5v31Hjqu03o7+deEbEKN8T+pZtl/lena1EY4sTUwKm1DC6u iB3yUxX9M3tL1UpckiJ03hAcUE4MpZn2jkaZ23VvPxN9Mpmb1PshzM7zrW9KA3Z/jnAftWtV1qu75UaV+mo+6iD3s45DmeYw6GmiRd4jPOIJz1bDiq3MuvtMtQq5ZhflvXwAQSBkB0=</latexit> B-ORG I-ORG B-NT I-NT Figure 1: Example of weakly labeled data. B-NT and I-NT denote incomplete labels without types. et al., 2016), which has became a basic architec- ture due to its superior performance. Nevertheless, NN-CRFs require exhaustive human efforts for training annotations, and may not perform well in low-resource settings (Ni et al., …

Content truncated to 500 words