MULTI-MODAL AMHARIC MOVIE GENRE CLASSIFICATION USING A DEEP LEARNING APPROACH

ABSTRACTThe rapid growth of Ethiopia's film industry, especially in Amharic cinema, has created a significant need for intelligent systems that can automatically categorize films into genres. Traditional genre classification methods, which mainly rely on manual input, lack scalability, time consumption, and are prone to inconsistency and human mistakes. Even if there are many research studies conducted on movie genre classification using machine learning and deep learning with multi-modal data, there has been no research conducted on classifying the genre of Amharic movies. To address these issues, we introduced a novel deep learning-based multimodal model for Amharic movie genre categorization that uses both video frames and auditory data from movie trailers. We used 10,152 frames of sequences and 13,169 chunked audio data for training the model. In this study, various preprocessing techniques, including Wiener filter and spectral subtraction for audio enhancement, as well as CLAHE and Gaussian filtering for frame enhancement, are employed. Furthermore, frame-level features were extracted using a pre-trained Inflated 3D ConvNet (I3D), while audio features were obtained through a hybrid approach that integrated hand-crafted descriptors, such as MFCC, ZCR, Chroma, spectral roll off, and RMSE, with deep features derived from a Bidirectional Long Short-term memory (BiLSTM) network. This study explores a deep learning-based approach to multimodal classification of movie genres, incorporating advanced neural network architectures such as I3D CNN, CNN, and Bi-LSTM, as well as CNN-BiLSTM for frame, audio, and fused data, respectively. The dataset is categorized as action, comedy, drama, and romance labels. A multiclass classification strategy, in which each category was treated independently, has been implemented. Next, models were built that can classify Amharic movie genres in a multi-class manner using deep learning algorithms and measure the model’s performance in terms of accuracy. Individual experiments were carried out for each modality, with feature-level integration performed through early fusion. Upon evaluating multiple hyperparameter settings, the models I3D, CNN, BiLSTM, and CNN-BiLSTM achieved accuracies of 89%, 77%, 82%, and 92%, respectively. In summary, the integration of frame and audio features through early fusion in the CNN-BiLSTM model yielded higher accuracy than the unimodal algorithms evaluated in this study. As a result, this study improves Amharic movie organization, simplifies movie discovery, and enhances recommendation systems. Keywords: 3D CNN, Amharic movie genre, Multi-Modal Amharic movie genre classification.