Unified Semantic Typing with Meaningful Label Inference

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2642 - 2654 July 10-15, 2022 ©2022 Association for Computational Linguistics Unified Semantic Typing with Meaningful Label Inference James Y. Huang, Bangzheng Li∗, Jiashu Xu∗and Muhao Chen University of Southern California Los Angeles, California, USA {huangjam,bangzhen,jiashuxu,muhaoche}@usc.edu Abstract Semantic typing aims at classifying tokens or spans of interest in a textual context into seman- tic categories such as relations, entity types, and event types. The inferred labels of seman- tic categories meaningfully interpret how ma- chines understand components of text. In this paper, we present UNIST, a unified framework for semantic typing that captures label seman- tics by projecting both inputs and labels into a joint semantic embedding space. To formu- late different lexical and relational semantic typing tasks as a unified task, we incorporate task descriptions to be jointly encoded with the input, allowing UNIST to be adapted to dif- ferent tasks without introducing task-specific model components. UNIST optimizes a mar- gin ranking loss such that the semantic related- ness of the input and labels is reflected from their embedding similarity. Our experiments demonstrate that UNIST achieves strong perfor- mance across three semantic typing tasks: en- tity typing, relation classification and event typ- ing. Meanwhile, UNIST effectively transfers semantic knowledge of labels and substantially improves generalizability on inferring rarely seen and unseen types. In addition, multiple semantic typing tasks can be jointly trained within the unified framework, leading to a sin- gle compact multi-tasking model that performs comparably to dedicated single-task models, while offering even better transferability.1 1 Introduction Semantic typing is a group of fundamental natu- ral language understanding problems that aim at classifying tokens (or spans) of interest into se- mantic categories. This includes a wide range of long-standing NLP problems such as entity typing, relation classification, and event typing. Inferring the types of entities, relations or events mentioned ∗Equal contributions. 1Our code and pre-trained models are available at https: //github.com/luka-group/UniST. is not only crucial to the structural perception of human language, but also plays an important role in many downstream tasks such as entity linking (Onoe and Durrett, 2020), information extraction (Zhong and Chen, 2021) and question answering (Yavuz et al., 2016). Most traditional methods tackle semantic typing problems by training task-specific multi-class clas- sifiers with token or sentence representations from language models to predict a probability distribu- tion over a pre-defined set of classes (Dai et al., 2021; Yamada et al., 2020). However, this ap- proach comes with several limitations. First, these models simply convert labels into indices, thus completely ignoring the rich semantics carried by the label text itself. For example, given “Currently Ritek is the largest producer of OLEDs.”, know- ing what the entity type company means would naturally simplify the inference of “Ritek” is a com- pany in this context. Second, models trained as classifiers do not generalize well to class labels that are rarely seen or unseen in the training data, as …

Content truncated to 500 words