Positional Encoding
Related Papers
On the Ability and Limitations of Transformers to Recognize Formal Languages
AbstractTransformers have supplanted recurrent models in a large number of NLP tasks. However, the differences in their abilities to model different syntactic properties remain largely unknown. Past works suggest that …...
A-TIP: Attribute-aware Text Infilling via Pre-trained Language Model
AbstractText infilling aims to restore incomplete texts by filling in blanks, which has attracted more attention recently because of its wide application in ancient text restoration and text rewriting. However, …...
Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework
AbstractSimultaneous machine translation (SiMT) starts translating while receiving the streaming source inputs, and hence the source sentence is always incomplete during translating. Different from the full-sentence MT using the conventional …...
S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversation
AbstractEmotion recognition in conversation (ERC) has attracted much attention in recent years for its necessity in widespread applications. With the development of graph neural network (GNN), recent state-of-the-art ERC models …...
Interpreting Positional Information in Perspective of Word Order
AbstractThe attention mechanism is a powerful and effective method utilized in natural language processing. However, it has been observed that this method is insensitive to positional information. Although several studies …...