ConvNext CNN and transformer architecture based video retrieval system to perform large scale video retrieval for a query video from a large video database
Due to the increasing popularity of social media platforms and enhanced ease of access to the Internet, there has been explosive growth in the quantity of video data being generated throughout the world in the present era. Thus, there is an urgent requirement for advanced techniques to perform content-based video retrieval in a fast, accurate and efficient manner from huge databases of video data. The existing video retrieval systems mainly function based on a two-step process of feature extraction and then video similarity comparison based on extracted features. But the existing systems do not leverage both the spatial and temporal features of video data at the same time in the feature extraction step, and the feature vector representations of videos are often too large, thus, making the video similarity comparison too inefficient when dealing with huge video databases. To tackle these challenges, this paper proposes a novel deep learning-based system to perform large- scale video retrieval. The proposed system uses sophisticated convolutional neural network (CNN) architectures to extract spatial features, transformer architectures to extract temporal features and hashing techniques to create compact binary repre- sentations for videos, thus allowing efficient and fast comparison. The system is trained using a triplet-loss-based objective on the JHMDB public video dataset, and evaluation on the same dataset reveals that the proposed system outperforms existing video retrieval systems in all metrics, thus, creating a new milestone in the field of large-scale content-based video retrieval.