VISION DIFFMASK: Interpretability of Computer Vision models with Differentiable Patch Masking

We developed a competitive post-hoc interpretation method for vision tasks, focusing on both faithfulness and plausibility. The method implements a minimal input subset selection latent variable model for Vision Transformer, preserving the predicted distribution over classes. We trained the explanation model to produce attributions that are interpretable in a formal mathematical sense.

Conference on Computer Vision and Pattern Recognition XAI4CV Workshop