Bridging Vision and Language: Contrastive vs Non-Contrastive Learning for Image-Caption Retrieval

In traditional retrieval systems, text and image queries are processed independently through unimodal methods. However, real-world applications necessitate cross-modal retrieval techniques for better effectiveness. This study proposes a contrastive learning-based approach that integrates BERT (Bidirectional Encoder Representations from Transformers) and Vision Transformer (ViT) for bidirectional image-caption retrieval. This method demonstrates state-of-the-art performance on the Flickr8K dataset, achieving BLEU and ROUGE-L scores of 0.9397 and 0.9487, respectively. Additionally, we introduce a non-contrastive counterpart utilizing mean squared error (MSE) loss for direct feature matching, which enhances computational efficiency while maintaining competitive performance and higher retrieval accuracy. Our comparative evaluation highlights that, while contrastive learning offers subtle semantic alignment, non-contrastive learning serves as an efficient, computation-driven alternative. These findings provide valuable insights into optimizing vision-language models.