Exhibiting transparency without opening the 'Black Box' - Balancing act between Data Protection and Trade Secrets Rights in Solely Automated Decision-Making AI system in Healthcare

What was once called science fiction has developed over the years to be one of the most strategic technologies of the 21st century – artificial intelligence (AI) is real. The rapid digitalization has opened new pathways in Swedish healthcare, by increasing productivity and the effectiveness of care delivery as well as helping more patients in receiving better care. Yet, when fully automated decision-making AI system is at stake, where medical decisions are delegated to an AI algorithm, a conflict between two rights arise – the right of the patient to a transparent processing of its data concerning health and the right of the healthcare provider to keep its AI algorithms used in automated processing as a trade secret. Since no medical decisions have been fully delegated to an AI algorithm within the Swedish healthcare, this thesis aims at examining the risks and opportunities of such situation. Patient’s data protection rights to a transparent processing of its data concerning heath in automated decision making are found in the General Data Protection Regulation (GDPR) and complementary Swedish legislation. These rights are mainly the notification obligations, the right to access and additional safeguards, according to which the patient has the right to receive and access the ‘meaningful information about the logic involved’ of such automated processing. On contrary, the trade secret protection of automated decision-making AI algorithms, makes it difficult for the healthcare provider to comply with their transparency obligations under the GDPR, due to the opaqueness of such algorithms, e.g. ‘black box’ issue. The analysis shows that although the formulation of the ‘meaningful information’ can be relied upon by the healthcare provider, because notion of ‘meaningful’ shall be determined from the perspective of the patient where they do not need to receive the mathematical explanation of the processing method, the GDPR still makes clear that trade secrets cannot be relied upon to refuse to provide all of the information to the patient. Consequently, when all of the ‘meaningful information’ cannot be provided to the patient without healthcare provider reveals some of its precious AI algorithms protected by trade secrets, the question thus arises – which of the conflicting rights prevails? By taking a closer look at the legislation protecting the rights in conflict, a preference for patient’s data protection rights is confirmed. Yet, the GDPR allows Member States to introduce national restrictions, where trade secret protection have a restricting factor on transparency rights of the patient. Additionally, due to the pressure from the regulators and the society, new approaches are being introduced by researchers and practitioners, which are further presented in the thesis. The thesis concludes that the future of AI requires a dialogue between developers and the society about not only what is possible, but also what is reasonable. But for now, transparency in automated AI systems continue to be in need for careful examination, both by Data Protection Authorities and the national courts, together with the European Court of Justice, to find a solution where transparency can be exhibited without opening up the ‘black box’