Self-Paced Curriculum Learning for Visual Question Answering on Remote Sensing Data

Abstract

Answering questions with natural language by extracting in-formation from image has great potential in various applica-tions. Although visual question answering (VQA) for naturalimage has been broadly studied, VQA for remote sensing datais still in the early research stage. For the same remote sens-ing image, there exist questions with dramatically differentdifficulty-levels. Treating these questions equally may mis-lead the model and limit the VQA model performance. Con-sidering this problem, in this work, we propose a self-pacedcurriculum learning (SPCL) based VQA model with hard andsoft weighting strategies for remote sensing data. Like humanlearning process, the model is trained from easy to hard ques-tion samples gradually. Extensive experimental results on twodatasets demonstrate that the proposed training method canachieve promising performance.