Параметры
Related Papers
SRI International Results February 1992 ATIS Benchmark Test
We describe the results that SRI International achieved on the February 1992 ATIS Speech and Natural Language Sys- tem Test. The basic architecture of the system is described, including a …...
A Distributional Analysis of a Lexicalized Statistical Parsing Model
This paper presents some of the first data visualiza- tions and analysis of distributions for a lexicalized statistical parsing model, in order to better under- stand their nature. In the …...
Exploiting Web-Derived Selectional Preference to Improve Statistical Dependency Parsing
In this paper, we present a novel approach which incorporates the web-derived selec- tional preferences to improve statistical de- pendency parsing. Conventional selectional preference learning methods have usually fo- cused …...
Discriminative Training of 150 Million Translation Parameters and Its Application to Pruning
Until recently, the application of discrimina- tive training to log linear-based statistical ma- chine translation has been limited to tuning the weights of a limited number of features or training …...
Neural Scoring Function for MST Parser
AbstractContinuous word representations appeared to be a useful feature in many natural language processing tasks. Using fixed-dimension pre-trained word embeddings allows avoiding sparse bag-of-words representation and to train models with …...
Simple and Effective Text Matching with Richer Alignment Features
AbstractIn this paper, we present a fast and strong neural approach for general purpose text matching applications. We explore what is sufficient to build a fast and well-performed text matching …...
Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translation
AbstractNeural machine translation (NMT) models usually suffer from catastrophic forgetting during continual training where the models tend to gradually forget previously learned knowledge and swing to fit the newly added …...
On the Sparsity of Neural Machine Translation Models
AbstractModern neural machine translation (NMT) models employ a large number of parameters, which leads to serious over-parameterization and typically causes the underutilization of computational resources. In response to this problem, …...
One Semantic Parser to Parse Them All: Sequence to Sequence Multi-Task Learning on Semantic Parsing Datasets
AbstractSemantic parsers map natural language utterances to meaning representations. The lack of a single standard for meaning representations led to the creation of a plethora of semantic parsing datasets. To …...
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts
AbstractThis work introduces a new multi-task, parameter-efficient language model (LM) tuning method that learns to transfer knowledge across different tasks via a mixture of soft prompts—small prefix embedding vectors pre-trained …...
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models
AbstractTransformer-based pre-trained language models (PLMs) mostly suffer from excessive overhead despite their advanced capacity. For resource-constrained devices, there is an urgent need for a spatially and temporally efficient model which …...
HashFormers: Towards Vocabulary-independent Pre-trained Transformers
AbstractTransformer-based pre-trained language models are vocabulary-dependent, mapping by default each token to its corresponding embedding. This one-to-one mapping results into embedding matrices that occupy a lot of memory (i.e. millions …...
Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure
AbstractWith the availability of massive general-domain dialogue data, pre-trained dialogue generation appears to be super appealing to transfer knowledge from the general domain to downstream applications. In most existing work, …...