Mutual Information for Multi-modal, Discontinuity-Preserving Image Registration

Abstract

Multi-sensory data fusion and medical image analysis often pose the challenging task of aligning dense, non-rigid and multi-modal images. However, optical sequences may also present illumination variations and noise. The above problems can be addressed by an invariant similarity measure, such as mutual information. However, in a variational setting convex formulations are generally recommended for efficiency reasons, especially when discontinuities at the motion boundaries have to be preserved. In this paper we propose the TV-MI approach, addressing for the first time all of the above issues, through a primal-dual estimation framework, and a novel approximation of the pixel-wise Hessian matrix, decoupling pixel dependencies while being asymptotically correct. At the same time, we keep a high computational efficiency by means of prequantized kernel density estimation and differentiation. Our approach is demonstrated on ground-truth data from the Middlebury database, as well as medical and visible-infrared image pairs.