Жасырын сөзді болжау

AbstractRecent works show that pre-trained language models (PTLMs), such as BERT, possess certain commonsense and factual knowledge. They suggest that it is promising to use PTLMs as “neural knowledge bases” …...

Cross-lingual pre-training aims at providing effective prior representations for the inputs from multiple languages. With the model- ing of bidirectional contexts, recently preva- lent language modeling approaches such as XLM …...

AbstractSpecialized number representations in NLP have shown improvements on numerical reasoning tasks like arithmetic word problems and masked number prediction. But humans also use numeracy to make better sense of …...