LSTMs
Related Papers
Towards Automating Healthcare Question Answering in a Noisy Multilingual Low-Resource Setting
AbstractWe discuss ongoing work into automating a multilingual digital helpdesk service available via text messaging to pregnant and breastfeeding mothers in South Africa. Our anonymized dataset consists of short informal …...
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
AbstractNatural language is characterized by compositionality: the meaning of a complex expression is constructed from the meanings of its constituent parts. To facilitate the evaluation of the compositional abilities of …...
How Can Self-Attention Networks Recognize Dyck-n Languages?
AbstractWe focus on the recognition of Dyck-n (Dn) languages with self-attention (SA) networks, which has been deemed to be a difficult task for these networks. We compare the performance of …...
Tensorized Embedding Layers
AbstractThe embedding layers transforming input words into real vectors are the key components of deep neural networks used in natural language processing. However, when the vocabulary is large, the corresponding …...
Theoretical Limitations of Self-Attention in Neural Sequence Models
AbstractTransformers are emerging as the new workhorse of NLP, showing great success across tasks. Unlike LSTMs, transformers process input sequences entirely through self-attention. Previous work has suggested that the computational …...