Мультимодальная предобучение

AbstractMultimodal pre-training has propelled great advancement in vision-and-language research. These large-scale pre-trained models, although successful, fatefully suffer from slow inference speed due to enormous computational cost mainly from cross-modal attention …...

We present a simplified, task-agnostic multi- modal pre-training approach that can accept either video or text input, or both for a va- riety of end tasks. Existing pre-training are task-specific …...

AbstractRecent research has made impressive progress in large-scale multimodal pre-training. In the context of the rapid growth of model size, it is necessary to seek efficient and flexible methods other …...