Content Based Video Retrieval using Deep Learning

Abstract Content-based video retrieval (CBVR) is part of a search engine which is widely utilized in everyday life for retrieving relevant images/frames from given large datasets. Generally, the existing methods don't give the relevance and expected images as per the query images and also require more computational time. The CBVR systems depend on the feature representation and similarity measurements. In this project, a CBVR system is developed using deep learning models. The convolutional neural network (CNN) is used to extract features from an input image/frame, and then comparing those features with images in datasets to retrieve similar images/frames. The process involves three main phases namely feature extraction, feature comparison, and retrieval. In feature extraction, the CNN is trained on a dataset of images to learn to extract features that are relevant to the task at hand. In feature comparison, the features extracted from the input image are compared to the features of the images in the database using a similarity measure such as Euclidean distance. Finally, in the retrieval phase, the most similar images/frames from the datasets are returned as the results. The CBVR is carried out using a pre-trained deep learning model, such as VGG16 or ResNet, which is used to train over large image datasets and can extract useful features from new images. In this project, the retrieval of images are carried out and the performance is evaluated based on accuracy, precision, recall. Various CCN models are experimented to perform result analysis. Keywords: Control based video retrieval, Deep learning, Convolutional Neural Network, Information Retrieval, Feature extraction, Inception V3, Feature selection