elib
DLR-Header
DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

A Novel Actor Dual-Critic Model for Remote Sensing Image Captioning

Chavhan, Ruchika and Banerjee, Biplab and Zhu, Xiao Xiang and Chaudhuri, Subhasis (2021) A Novel Actor Dual-Critic Model for Remote Sensing Image Captioning. ICPR 2020, 10.-15.1.21, Milan, Italy.

[img] PDF
2MB

Abstract

We deal with the problem of generating textual captions from optical remote sensing (RS) images using the notion of deep reinforcement learning. Due to the high inter-class similarity in reference sentences describing remote sensing data, jointly encoding the sentences and images encourages prediction of captions that are semantically more precise than the ground truth in many cases. To this end, we introduce an Actor Dual-Critic training strategy where a second critic model is deployed in the form of an encoder-decoder RNN to encode the latent information corresponding to the original and generated captions. While all actor-critic methods use an actor to predict sentences for an image and a critic to provide rewards, our proposed encoder-decoder RNN guarantees high-level comprehension of images by sentence-to-image translation. We observe that the proposed model generates sentences on the test data highly similar to the ground truth and is successful in generating even better captions in many critical cases. Extensive experiments on the benchmark Remote Sensing Image Captioning Dataset (RSICD) and the UCM-captions dataset confirm the superiority of the proposed approach in comparison to the previous state-of-the-art where we obtain a gain of sharp increments in both the ROUGE-L and CIDEr measures.

Item URL in elib:https://elib.dlr.de/139445/
Document Type:Conference or Workshop Item (Speech)
Title:A Novel Actor Dual-Critic Model for Remote Sensing Image Captioning
Authors:
AuthorsInstitution or Email of AuthorsAuthor's ORCID iD
Chavhan, RuchikaUNSPECIFIEDUNSPECIFIED
Banerjee, BiplabUNSPECIFIEDUNSPECIFIED
Zhu, Xiao Xiangxiao.zhu (at) dlr.deUNSPECIFIED
Chaudhuri, SubhasisUNSPECIFIEDUNSPECIFIED
Date:2021
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In SCOPUS:No
In ISI Web of Science:No
Page Range:pp. 1-8
Status:Accepted
Keywords:remote sensing, image captioning, text captions
Event Title:ICPR 2020
Event Location:Milan, Italy
Event Type:international Conference
Event Dates:10.-15.1.21
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Remote Sensing and Geo Research
Location: Oberpfaffenhofen
Institutes and Institutions:Remote Sensing Technology Institute > EO Data Science
Deposited By: Bratasanu, Ion-Dragos
Deposited On:18 Dec 2020 12:37
Last Modified:18 Dec 2020 12:48

Repository Staff Only: item control page

Browse
Search
Help & Contact
Information
electronic library is running on EPrints 3.3.12
Copyright © 2008-2017 German Aerospace Center (DLR). All rights reserved.