Flickr30k
Related Papers
OSU Multimodal Machine Translation System Report
This paper describes Oregon State Uni- versity’s submissions to the shared WMT’17 task “multimodal translation task I”. In this task, all the sentence pairs are image captions in different languages. …...
Learning to Represent Image and Text with Denotation Graph
AbstractLearning to fuse vision and language information and representing them is an important research problem with many applications. Recent progresses have leveraged the ideas of pre-training (from language modeling) and …...
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding
AbstractPhrase localization is a task that studies the mapping from textual phrases to regions of an image. Given difficulties in annotating phrase-to-object datasets at scale, we develop a Multimodal Alignment …...
Hashing based Efficient Inference for Image-Text Matching
Image-text matching has been a popular re- search topic which bridges vision and lan- guage through semantic understanding. Re- cent works mainly focus on exploring the in- teractions between images …...
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
AbstractMultimodal pre-training has propelled great advancement in vision-and-language research. These large-scale pre-trained models, although successful, fatefully suffer from slow inference speed due to enormous computational cost mainly from cross-modal attention …...
VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words
AbstractText-to-image retrieval is an essential task in cross-modal information retrieval, i.e., retrieving relevant images from a large and unlabelled dataset given textual queries. In this paper, we propose VisualSparta, a …...
Building Joint Relationship Attention Network for Image-Text Generation
AbstractAttention based methods for image-text generation often focus on visual features individually, while ignoring relationship information among image features that provides important guidance for generating sentences. To alleviate this issue, …...
Cross-Modal Similarity-Based Curriculum Learning for Image Captioning
AbstractImage captioning models require the high-level generalization ability to describe the contents of various images in words. Most existing approaches treat the image–caption pairs equally in their training without considering …...
Image Caption Generation for Low-Resource Assamese Language
AbstractImage captioning is a prominent Artificial Intelligence (AI) research area that deals with visual recognition and a linguistic description of the image. It is an interdisciplinary field concerning how computers …...
Leaner and Faster: Two-Stage Model Compression for Lightweight Text-Image Retrieval
AbstractCurrent text-image approaches (e.g., CLIP) typically adopt dual-encoder architecture using pre-trained vision-language representation. However, these models still pose non-trivial memory requirements and substantial incremental indexing time, which makes them less …...
Improving Visual-Semantic Embedding with Adaptive Pooling and Optimization Objective
AbstractVisual-Semantic Embedding (VSE) aims to learn an embedding space where related visual and semantic instances are close to each other. Recent VSE models tend to design complex structures to pool …...
ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning
AbstractTwo-Tower Vision-Language (VL) models have shown promising improvements on various downstream VL tasks. Although the most advanced work improves performance by building bridges between encoders, it suffers from ineffective layer-by-layer …...