Towards Explaining Automated Credit Decisions: The design of an Explicability Assessment Framework (EAF) for Machine Learning Systems

The use of machine learning systems has great potential to better predict probabilities of default for credit underwriting. Despite this advantage, herewith there exists the substantial risk of discrimination. Moreover, machine learning models with the highest prediction-accuracy are often the least explicable (i.e. explainable). Nonetheless, explicability is needed to create accountability of automated credit decisions by machine learning systems. Furthermore, there exists a regulatory need for explicability of machine learning systems in the General Data Protection Regulation (GDPR) and the Consumer Credit Directive (CCD). Besides that, an ethical- and societal need exists for explicability. Within the exploration of literature, it becomes clear that research lacks on how to move from a high-level principle like explicability, towards a prospective assessment of a machine learning use case on this principle, it lacks a multi-disciplinary perspective, and it misses an assessment framework that can guide decision-makers within machine learning use cases, aligned with a multi-organizational development lifecycle. This research aims to design a prospective pragmatic assessment framework that can guide decision-makers, within machine learning applications in European credit underwriting cases from the point of view of explicability. To accomplish this, the Design Science Research Methodology (DSRM), complemented with the Value Sensitive Design (VSD) approach, is utilized. To this end, the Explicability Assessment Framework (EAF) was developed. This framework is adapted to the context- and explanation characteristics of the case, and aligns with the CRISP-DM development lifecycle. It was found in two case studies that the framework helps with the decision-making whether a machine learning system is sufficiently explicable or not. Lastly, a wide range of future research areas is identified that needs attention: empirical validation and expansion of the framework, the relevance for automated explanation creation, the scalability to other context and a large amount of explanations, and the practical perspective regarding adoption in the industry.Complex Systems Engineering and Management (CoSEM