Жылдамдату

We describe a parallel implementation of a chart parser for a shared-memory multipro- cessor. The speed-ups obtained with this parser have been measured for a number of small natural-language grammars. …...

This paper describes new and improved tech- niques which help a unification-based parser to process input efficiently and robustly. In com- bination these methods result in a speed-up in parsing …...

Best-first chart parsing utilises a figure of merit (FOM) to efficiently guide a parse by first attending to those edges judged better. In the past it has usually been static; …...

Training word alignment models on large cor- pora is a very time-consuming processes. This paper describes two parallel implementations of GIZA++ that accelerate this word align- ment process. One of …...

Recent speed-ups for training large-scale models like those found in statistical NLP exploit distributed computing (either on multicore or “cloud” architectures) and rapidly converging online learning algo- rithms. Here we …...

A re-scoring strategy is proposed that makes it feasible to capture more long-distance de- pendencies in the natural language. Two pass strategies have become popular in a num- ber of …...

Conditional random fields and other graphi- cal models have achieved state of the art re- sults in a variety of tasks such as coreference, relation extraction, data integration, and pars- …...

A precise syntacto-semantic analysis of En- glish requires a large detailed lexicon with the possibility of treating multiple tokens as a sin- gle meaning-bearing unit, a word-with-spaces. However parsing with …...

AbstractSemantic parsers conventionally construct logical forms bottom-up in a fixed order, resulting in the generation of many extraneous partial logical forms. In this paper, we combine ideas from imitation learning …...

AbstractWe present a framework for question answering that can efficiently scale to longer documents while maintaining or even improving performance of state-of-the-art models. While most successful approaches for reading comprehension …...

AbstractTo learn a semantic parser from denotations, a learning algorithm must search over a combinatorially large space of logical forms for ones consistent with the annotated denotations. We propose a …...

AbstractWe make distributed stochastic gradient descent faster by exchanging sparse updates instead of dense updates. Gradient updates are positively skewed as most updates are near zero, so we map the …...

AbstractContextual representation models have achieved great success in improving various downstream natural language processing tasks. However, these language-model-based encoders are difficult to train due to their large parameter size and …...

AbstractHuman language is a rich multimodal signal consisting of spoken words, facial expressions, body gestures, and vocal intonations. Learning representations for these spoken utterances is a complex research problem due …...

AbstractPre-trained models like BERT ((Devlin et al., 2018) have dominated NLP / IR applications such as single sentence classification, text pair classification, and question answering. However, deploying these models in …...

AbstractPre-trained language models like BERT have proven to be highly performant. However, they are often computationally expensive in many practical scenarios, for such heavy models can hardly be readily implemented …...

AbstractPrior research notes that BERT’s computational cost grows quadratically with sequence length thus leading to longer training times, higher GPU memory constraints and carbon emissions. While recent work seeks to …...

AbstractRecent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity. However, the extra latency and memory costs introduced by this approach may …...

AbstractWe present ReadOnce Transformers, an approach to convert a transformer-based model into one that can build an information-capturing, task-independent, and compressed representation of text. The resulting representation is reusable across …...

AbstractNon-autoregressive translation (NAT) model achieves a much faster inference speed than the autoregressive translation (AT) model because it can simultaneously predict all tokens during inference. However, its translation quality suffers …...

AbstractState-of-the-art neural models typically encode document-query pairs using cross-attention for re-ranking. To this end, models generally utilize an encoder-only (like BERT) paradigm or an encoder-decoder (like T5) approach. These paradigms, …...

AbstractThe growing size of neural language models has led to increased attention in model compression. The two predominant approaches are pruning, which gradually removes weights from a pre-trained model, and …...

AbstractReal-world applications of neural language models often involve running many different models over the same corpus. The high computational cost of these runs has led to interest in techniques that …...