Adaptive Multi-source Domain Collaborative Fine-tuning for Transfer Learning

Fine-tuning is an important technique in transfer learning that has achieved significant success in tasks that lack training data. However, as it is difficult to extract effective features for single-source domain fine-tuning when the data distribution difference between the source and the target domain is large, we propose a transfer learning framework based on multi-source domain called adaptive multi-source domain collaborative fine-tuning to address this issue. This approach utilizes multiple source domain models for collaborative fine-tuning, thereby improving the model's feature extraction capability in the target task. Specifically, this approach employs an adaptive multi-source domain layer selection strategy to customize appropriate layer fine-tuning schemes for the target task among multiple source domain models. The objective is to extract more efficient features. Furthermore, a novel multi-source domain collaborative loss function is designed to facilitate the precise extraction of target data features by each source domain model. Simultaneously, it works towards minimizing the output difference among various source domain models, thereby enhancing the adaptability of the source domain model to the target data. In order to validate the effectiveness of the proposed the adaptive multi-source domain collaborative fine-tuning framework, it is analyzed and compared with the most widely used fine-tuning methods by applying them to seven publicly available visual categorization datasets commonly employed in transfer learning. Experimental results demonstrate that, in comparison with the existing fine-tuning approaches, our method not only enhances the accuracy of feature extraction in the model but also provides precise layer fine-tuning schemes for the target task, resulting in state-of-the-art performance.