Inducing Universal Semantic Tag Vectors

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3121–3127 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3121 Inducing Universal Semantic Tag Vectors Da Huo, Gerard de Melo Department of Computer Science, Rutgers University–New Brunswick, NJ, USA dh637@scarletmail.rutgers.edu, gdm@demelo.org Abstract Given the well-established usefulness of part-of-speech tag annotations in many syntactically oriented downstream NLP tasks, the recently proposed notion of semantic tagging (Bjerva et al., 2016) aims at tagging words with tags informed by semantic distinctions, which are likely to be useful across a range of semantic tasks. To this end, their annotation scheme distinguishes, for instance, privative attributes from subsective ones. While annotated corpora exist, their size is limited and thus many words are out-of-vocabulary words. In this paper, we study to what extent we can automatically predict the tags associated with unseen words. We draw on large-scale word representation data to derive a large new Semantic Tag lexicon. Our experiments show that we can infer semantic tags for words with high accuracy both monolingually and cross-lingually. Keywords: semantic tagging, word vectors, multilingual 1. Introduction Lexical category distinctions have been studied since the beginnings of linguistics. Traditional part-of-speech tag- ging has focused on distinctions based on the grammatical function of words, i.e., the syntactic role that words play within a sentence. Semantic Tags. In contrast, the recently proposed task of Semantic Tagging (Bjerva et al., 2016) considers a set of tags that are informed by semantic distinctions conjectured to be pertinent for semantic parsing and other semantics- driven tasks. The annotation scheme distinguishes, for instance, priva- tive attributes (PRI) such as former, fake from intersective ones (IST) such as vegetarian, and subsective ones (SST) such as skillful, making it easier for a system to discern that a fake detective is not a detective. Different kinds of named entities are distinguished (e.g., people, geopolitical entities, artifacts, etc.), as in named entity recognition, in- cluding temporal categories (e.g., there are separate tags for decades, day of week, etc.). Moreover, there are dedicated tags for different tenses of events (such as past tense ate), tense & aspect markers (e.g., was in was reading), roles (victim), implications (if, unless), greetings (hi, bye), and many others. At the same time, more syntactically moti- vated distinctions, such as between adjectives and adverbs, are disregarded in this annotation scheme. In follow-up work, Abzianidze and Bos (2017) presented an improved tag set and showed that the tags exhibit the potential to ap- ply cross-lingually. Motivation. In terms of available data, the Parallel Mean- ing Bank project (Abzianidze et al., 2017) provides a par- allel corpus covering four languages (English, Dutch, Ger- man, and Italian) with rich annotations based on Discourse Representation Theory. At the lexical level, it includes se- mantic tag annotations, which we rely on in our work. However, due to the novelty of this task, the available anno- tations are limited in quantity and consist of a mix of gold and silver standard data. Just …

Content truncated to 500 words