A Rank-Based Wasserstein Distance for Comparing the Contrastive Power of Post-Hoc XAI Techniques

Kurzfassung

Despite their widespread adoption in image classification, saliency map-based post-hoc explanation methods lack rigorous quantitative evaluation of their ability to capture contrastive, class-dependent decision structures. Existing evaluation techniques rarely assess whether explanations meaningfully differentiate between opposing class predictions, particularly under adversarial perturbations, where perceptually similar inputs yield substantial prediction shifts. Framing image classification as a deductively closed process, we argue that explanations must justify the predicted class while meaningfully excluding alternatives. To address this gap, we introduce a novel metric for quantifying the contrastive power of saliency map-based explanations -- a rank-based Wasserstein distance defined over saliency maps of original inputs and their adversarial counterparts. Our analysis reveals that popular saliency methods yield invariant explanations across divergent class predictions, indicating limited sensitivity to class-specific decision boundaries. This limitation is effectively captured by the proposed metric, positioning contrastive evaluation as an important criterion and valuable extension to existing explanation sanity checks.