Visual Question Answering System with Improved Question Understanding and Appropriate Answer Selection Using FLR-CRF

The recent work in Visual Question Answering (VQA) System ventures a synergy between the Vision and Language. These two constituents of Artificial Intelligence make it possible to answer the Questions from a given Image. This enhances Machine Intelligence thus paves a way for Computer Vision. The VQA system has reached a certain height with many state-of-art systems but still there exists a challenge in every application. The Visual Question Answering System is expected to describe the image first, then analyze the Natural Language Question being asked and finally map the answer for the question from the image descriptions. For image description Beam search and Diverse Beam search is used so far to capture the descriptions of the image and using CNN, RNN, LSTM the questions answering system is being developed. But still the system needs an improvement in the caption generation of the entire image and exact understanding of the question and selection of correct answer. The accurate mapping of the answers for the question is still an on demand research work. Hence this work is proposed to exactly i) Identify the question by using Fuzzy Rules by extracting the desired features and categorize the captions generated from the image into relevant and irrelevant classes using cosine similarity ii) Finally map the question to the answer from the image by Conditional Random Field, by summing the discriminative and translation probabilities which helps to extract the correct answer for the question being asked from the image. This model outperforms many of the state-of-art methods & provides a dynamic and skillful framework to answer the Questions asked from an image with an accuracy of 89.03%.