Teex: a toolbox for the evaluation of explanations

In the machine learning (ML) community, models are developed, trained and deployed for many applications. Text-to-speech, product and media recommendation, medical aiding, environmental protection and many more are examples of current ML applications. But, more often than not, given the quality requirements for the applications, these models can become very complex. So complex, in fact, that the decisions they take are usually not understandable by humans. These are called black box models. So, given the clear problem of not trusting models' decisions because of the rele- vance of their impact and their low transparency, explanation methods / explainers were born with the objective of distilling the factors that black box models take into account when making decisions into 'explanations', which humans can understand. There are many categorizations into which explanation methods fall. For example, the type of explanations they produce, on which models do they work, their mechanisms for extracting information or if they try to characterize a model's whole behaviour (global explanations) or individual predictions (local explanations). Given the current rise of the field of Explainable AI (XAI), which is driven by necessity, researchers need a tool to easily and swiftly evaluate the performance of state-of-the-art explainer methods. On top of current evaluation techniques such as performing subjective human experiments or manually comparing the quality of explanations, we present a toolbox that will allow to add another layer of credibility to part of XAI research. The toolbox is aimed at the automatic evaluation of local explanations via comparison to ground-truth explanations. Version 1.0 contains several evaluation metrics for different explanation types: saliency maps, decision rules and feature and word importance vectors. Moreover, the library also provides real-world and artificial data with available ground truth explanations so that users can easily benchmark local explainer methods