Contextual Language Models
Related Papers
Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining
AbstractRST-based discourse parsing is an important NLP task with numerous downstream applications, such as summarization, machine translation and opinion mining. In this paper, we demonstrate a simple, yet highly accurate …...
When is a bishop not like a rook? When it’s like a rabbi! Multi-prototype BERT embeddings for estimating semantic relationships
AbstractThis paper investigates contextual language models, which produce token representations, as a resource for lexical semantics at the word or type level. We construct multi-prototype word embeddings from bert-base-uncased (Devlin …...
ConRPG: Paraphrase Generation using Contexts as Regularizer
AbstractA long-standing issue with paraphrase generation is the lack of reliable supervision signals. In this paper, we propose a new unsupervised paradigm for paraphrase generation based on the assumption that …...
Scalar Adjective Identification and Multilingual Ranking
AbstractThe intensity relationship that holds between scalar adjectives (e.g., nice < great < wonderful) is highly relevant for natural language inference and common-sense reasoning. Previous research on scalar adjective ranking …...
Using Distributional Principles for the Semantic Study of Contextual Language Models
Many studies were recently done for investi- gating the properties of contextual language models but surprisingly, only a few of them consider the properties of these models in terms of …...