Ethical Approaches to Robotic Data Gathering in Academic Research

Internet researchers increasingly have at their disposal of an array of automated software agents, or “bots,” which can rapidly and efficiently retrieve a variety of economic and technical data from publicly accessible web sites. While these automated tools greatly facilitate the retrieval and analysis of data for academic research, they may pose ethical problems for Internet researchers. Specifically, automated software bots place some load on servers being accessed, possibly in contradiction to the expected use of such servers, and possibly in violation of the legal prerogatives of web site owners. Determining how and when to access such web sites, and whether to seek the consent of web site owners for retrieval of publicly accessible data presents an apparent conflict between general principles of information policy and the emerging legal precedent regarding trespass to computers. This conflict may be characterized as pitting utilitarian considerations against deontological considerations in a fashion reminiscent of previous debates over informed consent in on-line research. In this paper, we examine both utilitarian and deontological characterizations of the ethical obligations of researchers employing automated data retrieval bots, and argue that the contrasts between the two approaches do not necessarily result in conflict. Instead, we argue that the tension within the relevant practices indicates the need for a “meta-choice” between utilitarian and deontological considerations. We further suggest certain factors that may differentiate such a “metaethical” choice in the context of automated data retrieval from the “meta-ethical” choice presented in previously identified contexts of human subjects research or of web browser technology design. In the end, we argue that by analyzing the ethical issues in terms of the contrast between utilitarian and deontological ethics, it is possible to resolve some of the ethical dilemmas regarding automated data retrieval in fruitful and cogent ways