Есептеу құны
Related Papers
Fast Semantic Extraction Using a Novel Neural Network Architecture
We describe a novel neural network archi- tecture for the problem of semantic role la- beling. Many current solutions are compli- cated, consist of several stages and hand- built features, …...
Efficient Sequence Learning with Group Recurrent Networks
AbstractRecurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation, speech recognition and so on. One of the key factors to …...
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
AbstractThe state of the art on many NLP tasks is currently achieved by large pre-trained language models, which require a considerable amount of computation. We aim to reduce the inference …...
Obtaining Better Static Word Embeddings Using Contextual Embedding Models
AbstractThe advent of contextual word embeddings — representations of words which incorporate semantic and syntactic information from their context—has led to tremendous improvements on a wide variety of NLP tasks. …...
bert2BERT: Towards Reusable Pretrained Language Models
AbstractIn recent years, researchers tend to pre-train ever-larger language models to explore the upper limit of deep models. However, large language model pre-training costs intensive computational resources, and most of …...
Free Lunch for Efficient Textual Commonsense Integration in Language Models
AbstractRecent years have witnessed the emergence of textual commonsense knowledge bases, aimed at providing more nuanced and context-rich knowledge. The integration of external commonsense into language models has been shown …...
Plug-and-Play Document Modules for Pre-trained Models
AbstractLarge-scale pre-trained models (PTMs) have been widely used in document-oriented NLP tasks, such as question answering. However, the encoding-task coupling requirement results in the repeated encoding of the same documents …...