Слои трансформеров
method
ID: transformerlayers
Related Papers
Probing Pretrained Language Models for Lexical Semantics
AbstractThe success of large pretrained language models (LMs) such as BERT and RoBERTa has sparked interest in probing their representations, in order to unveil what types of knowledge they implicitly …...
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
AbstractIn this paper, we study machine reading comprehension (MRC) on long texts: where a model takes as inputs a lengthy document and a query, extracts a text span from the …...
Graph-Based Multilingual Label Propagation for Low-Resource Part-of-Speech Tagging
AbstractPart-of-Speech (POS) tagging is an important component of the NLP pipeline, but many low-resource languages lack labeled data for training. An established method for training a POS tagger in such …...