Contextualized Models

We propose the first probabilistic approach to modeling cross-lingual semantic sim- ilarity (CLSS) in context which requires only comparable data. The approach re- lies on an idea of projecting words …...

AbstractOne of the most powerful features of contextualized models is their dynamic embeddings for words in context, leading to state-of-the-art representations for context-aware lexical semantics. In this paper, we present …...

AbstractSimilarity measures are a vital tool for understanding how language models represent and process language. Standard representational similarity measures such as cosine similarity and Euclidean distance have been successfully used …...

For domain-specific NLP tasks, applying word embeddings trained on general cor- pora is not optimal. Meanwhile, training domain-specific word representations poses challenges to dataset construction and embed- ding evaluation. In …...

AbstractThis paper presents a multilingual study of word meaning representations in context. We assess the ability of both static and contextualized models to adequately represent different lexical-semantic relations, such as …...

AbstractState-of-the-art pretrained contextualized models (PCM) eg. BERT use tasks such as WiC and WSD to evaluate their word-in-context representations. This inherently assumes that performance in these tasks reflect how well …...