SyntStereo2Real: Edge aware GAN for remote sensing image translation while maintaining stereo constraints

Kurzfassung

In the field of remote sensing, the scarcity of stereo-matched data often hinders the training of deep neural networks. The use of synthetically generated images as an alternative alleviates this difficulty but suffers from the problem of domain generalization. Unifying the capabilities of image-to-image translation and stereo-matching presents an effective solution to address the problem of domain generalization. Current methods involve combining two networks—an unpaired image-to-image translation network and a stereo-matching network—while jointly optimizing them. This work proposes a single edge-aware GAN-based network that effectively tackles both tasks simultaneously. We obtain edge maps of input images from the sobel operator and use it as an additional input to the encoder in the generator to enforce geometric consistency during translation. Additionally, we include a warping loss from translated images to maintain the stereo consistency. This work performs qualitatively and quantitatively better than existing models, and its applicability extends to diverse domains, including autonomous driving.