Towards an integrated evaluation framework for xai: an experimental study

Increasing prevalence of opaque black-box AI has highlighted the need for explanations of their behaviours, for example, via explanation artefacts/proxy models. The current paper presents a paradigm for human-grounded experiments to evaluate the relationship between explanation fidelity, human learning performance, understanding and trust in a black-box AI by manipulating the complexity of an explanatory artefact. Decision trees were used in the current experiment as exemplar interpretable surrogate models, providing explanations approximating black-box behaviour, by means of explanation by simplification. Consistent with our hypotheses: 1) explanatory artefacts brought about better learning, while greater decision tree depths led to greater interpretability of the AI's performance and greater trust in the AI; and 2) explanatory artefacts facilitated learning and task performance even after they were withdrawn. Findings are discussed in terms of the interplay between human understanding, trust and AI system performance, highlighting the simplifying assumption of a monotonic relationship between explanation fidelity and interpretability