A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering

Presentation video of the paper titled ''A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Question Answering.'' This paper introduces a novel pipeline for Knowledge-Intensive Visual Question Answering (KI-VQA) tasks, comprising a retriever (DEDR) and a reader (MM-FiD). DEDR utilizes a dual encoding dense retrieval framework, incorporating both textual and multi-modal encoders, and employs an iterative knowledge distillation approach to align their representation spaces. Extensive evaluation of OK-VQA and FVQA datasets demonstrates that DEDR outperforms state-of-the-art baselines by 11.6% and 30.9%, respectively. MM-FiD, leveraging the retrieved passages, utilizes an encoder-decoder multi-modal fusion-in-decoder model to generate textual answers, surpassing competitive baselines by 5.5% on OK-VQA and 8.5% on FVQA in terms of question answering accuracy.