Collaborative research-grade software for crowd-sourced data exploration: from context to practice - Part I: Guidelines for scientific evidence provision for policy support based on Big Data and open technologies

The scope and focus of the research reported in this document is to implement a collaborative high-level research-grade application software platform for scientific experimentation and data analysis. By doing so, we aim at exploring, extracting value from, and making sense of massive, interconnected datasets. Namely, the software is designed as an application layer that makes use of suitable statitiscal, exploratory or descriptive techniques, as well as visualisation tools, in order to produce reasonable interpretations of data – e.g. consisting in crowd-sourced data from social media, as well as other domain-orientated data, like sensor-based and geospatial data – that are logical but not definitive in their claims. We believe that by starting small and building quickly through a pilot, and by gaining experience from its deployment, it is possible to foster interdisciplinary and collaborative research that conjoins domain expertise. All together, it will lead to more holistic and extensive approach of entire complex systems. In order to consider all the potential issues and address all possible challenges in the future implementation, we adopt a multi-stage approach that aims to first acquire a clear vision of how to use data analysis and analytics, and thereafter this vision to the strategic needs of our research institution. Part I of the report provides the current Big Data and open technologies context (landscapes) in terms of European policies, states the motivation for our approach, and the foundations for an open, verifiable, reproducible, collaborative, and participatory framework for its deployment, and formulates applicable recommendations for implementation. Indeed, while it relies mainly on secondary literature – on Big Data, open technologies and data-driven decision making, as well as policy documents – this report actually defines a set of practical guidelines for the deployment and implementation of a Big Data application software solution in our institution.JRC.H.6-Digital Earth and Reference Dat