Breaking the wall: Building an infrastructure to enable multi-disciplinary analyses for social sciences and the Internet of Things (IoT)

The UK Data Service has been investing in a unified repository infrastructure (DSaaP -- Data Services as a Platform) that enables true domain-agnostic research. Starting from first principles at the datum level, we have worked on the proposition that the richest impact on policy is generated when analysis can be conducted in one place on multiple domains (in this case social sciences and energy, although the principle is extensible to any domain). Both "big data" from household smart meters and traditional social science surveys can now be explored, linked and analysed in a single user interface, in a single infrastructure, referencing a single architecture and a single metadata model. Traditionally, this kind of "collective intelligence" has been difficult to achieve without expensive and resource-intensive data brokerage services which are hopelessly unscalable. We demonstrate a real-world portal where you can link streaming smart meter energy data and traditional survey data to create derived data products in real-time and explain how this fundamentally alters the way we enable cross-disciplinary research