Jaeger: A Concatenation-Based Multi-Transformer VQA Model
This addresses efficiency and accuracy issues in document-based VQA, though it appears incremental as it builds on existing transformer models with concatenation and dimensionality reduction.
The paper tackles the problem of slow response times and imprecision in document-based visual question answering by proposing Jaeger, a concatenation-based multi-transformer model that uses RoBERTa and GPT-2 for feature extraction. The model achieves competitive performance on Task C of the PDF-VQA Dataset while reducing computational cost and inference time.
Document-based Visual Question Answering poses a challenging task between linguistic sense disambiguation and fine-grained multimodal retrieval. Although there has been encouraging progress in document-based question answering due to the utilization of large language and open-world prior models\cite{1}, several challenges persist, including prolonged response times, extended inference durations, and imprecision in matching. In order to overcome these challenges, we propose Jaegar, a concatenation-based multi-transformer VQA model. To derive question features, we leverage the exceptional capabilities of RoBERTa large\cite{2} and GPT2-xl\cite{3} as feature extractors. Subsequently, we subject the outputs from both models to a concatenation process. This operation allows the model to consider information from diverse sources concurrently, strengthening its representational capability. By leveraging pre-trained models for feature extraction, our approach has the potential to amplify the performance of these models through concatenation. After concatenation, we apply dimensionality reduction to the output features, reducing the model's computational effectiveness and inference time. Empirical results demonstrate that our proposed model achieves competitive performance on Task C of the PDF-VQA Dataset. If the user adds any new data, they should make sure to style it as per the instructions provided in previous sections.