"M4: A Unified XAI Benchmark for Faithfulness Evaluation of Feature Att" by Xuhong Li, Mengnan Du et al.
 

M4: A Unified XAI Benchmark for Faithfulness Evaluation of Feature Attribution Methods across Metrics, Modalities and Models

Document Type

Conference Proceeding

Publication Date

1-1-2023

Abstract

While Explainable Artificial Intelligence (XAI) techniques have been widely studied to explain predictions made by deep neural networks, the way to evaluate the faithfulness of explanation results remains challenging, due to the heterogeneity of explanations for various models and the lack of ground-truth explanations. This paper introduces an XAI benchmark named M4, which allows evaluating various input feature attribution methods using the same set of faithfulness metrics across multiple data modalities (images and texts) and network structures (ResNets, MobileNets, Transformers). A taxonomy for the metrics has been proposed as well. We first categorize commonly used XAI evaluation metrics into three groups based on the ground truth they require. We then implement classic and state-of-the-art feature attribution methods using InterpretDL and conduct extensive experiments to compare methods and gain insights. Extensive experiments have been conducted to provide holistic evaluations as benchmark baselines. Several interesting observations are made for designing attribution algorithms. The implementation of state-of-the-art explanation methods and evaluation metrics of M4 is publicly available at https://github.com/PaddlePaddle/InterpretDL.

Identifier

85182713567 (Scopus)

ISBN

[9781713899921]

Publication Title

Advances in Neural Information Processing Systems

ISSN

10495258

Volume

36

Grant

2021ZD0110303

Fund Ref

National Key Research and Development Program of China

This document is currently not available here.

Share

COinS