Data sharing and personal privacy in contemporary public services: the social dynamics of ethical decision making

Amongst some of the most important and interesting ethical dilemmas facing street level bureaucrats in contemporary public services are those arising from conflicting imperatives in the use of personal data. On the one hand, public services are coming under pressure to retain and share more data about identifiable individuals, in order better to deal with their problems or to protect communities against the risks they pose. This pressure appears to conflict – at least to some degree - with confidentiality norms embedded in the codes of practice of public service professions as well as with privacy laws stemming from the European Data Protection Directive and the European Convention of Human Rights. Furthermore, the ethical dilemmas associated with these conflicting imperatives may be growing more acute, as a result of changes in the political and social environment in which public servants work. Firstly, there is a widespread perception that information and communication technologies can support the extensive networking of public service data systems: this perception is giving rise to pressures to achieve service improvements and cost savings associated with the pooling, re-use and exchange of personal data. Secondly, there is a growing view that many of problems experienced by individuals, families and very small neighbourhoods can best be addressed by multi-agency interventions: this view implies that agencies will share data about these individuals, families and neighbourhoods to a greater degree than hitherto. Thirdly, growing pressures on public services associated the influence of communitarian ideas about the management of risks may be leading to tendencies to favour the public good over individual rights, especially in such fields as policing, child protection, mental health and public health. If so, we would expect these pressures to lower thresholds for sharing personal data between agencies. This paper presents some provisional findings from a major research project funded by the UK’s Economic and Social Research Council. The project has collected qualitative data from over 200 interviews with street level professional workers, managers and information systems managers in 12 cases of local multi-agency arrangements (MAAs) in England and Scotland. The data presented in the present paper is from the 8 English cases, comprising 138 interviews. These cases were chosen from four policy fields, namely: • health and social care for the elderly • health and social care for the mentally ill • public protection arrangements managing risks associated with violent criminals and sex offenders; and, • crime and disorder reduction partnerships, which include organisations concerned with planning interventions against prolific offenders, domestic violence and drug-related crime. These fields have been chosen, for two main reasons. First, in all of these fields, decisions about what data to share, when to share them, who to share them with and how to interpret them and use them involve serious risks: the decisions made by individual workers may result in the abuse or death of a child, the loss of parole for a prisoner, the stigmatisation of a family or the refusal of employment for a job applicant. Decision-making in these fields therefore poses ethical problems with potentially serious outcomes for individual clients. Second, in the UK, all these fields are currently subject to central government initiatives designed to encourage greater sharing of personal data to support more effective multi-agency working. They are all fields, then, in which tensions with privacy are coming to the fore. The data collected for this project will eventually provide the most comprehensive, detailed evidence yet available about the ways in which street level professional workers cope on a day to day basis with the tensions between imperatives to share data about needy and risky people, and imperatives to respect their confidentiality and personal privacy. The data will also provide evidence about the ways in which the coping strategies of such workers may be changing under the influence of changes in the political environment outlined above. A particular facet of the analysis will be concerned with the intended and unintended behavioural consequences of the growing use of data sharing protocols and other ethical instruments. These instruments are designed to govern the practices of street level professionals, and in so doing to protect the privacy of clients, patients, offenders, victims, witnesses and other individuals who come into contact with public services in these fields. The overarching hypothesis framing this research is that individual decision-making will be shaped by the organisational, cultural dynamics in which it takes place. We are using neo-Durkheimian institutional theory as the analytical framework for a series of systematic comparisons: between MAAs in the four 3 different policy fields: between types of organisation (for example, police, health and social work agencies), between organisations that comprise these MAAs and between actors from different professions. These comparisons will enable us to assess the nature and influence of organisational dynamics in these fields, and to understand the ways that different mixes of institutional forms impinge on data sharing practices in different organisations and among different kinds of professional workers. We will also compare the ways in which risks to privacy are perceived and managed, and the ways organisational dynamics shape coping mechanisms adopted by individuals to manage the fear of blame. In turn, this analysis will help us understand the social influences on complex decision making by street level workers in policy fields that that are riven with important ethical issues