Immersive geometry-based and image-based exploration of cultural heritage models

Recent advances in 3D acquisition technologies have facilitated the inexpensive digitization of cultural heritage. In addition to the 3D digital model, in many cases multiple photo collections are also available. These photo collections often provide valuable information not included in the 3D digital model. In this paper we describe a VR-ready web application to simultaneously explore a cultural heritage model together with arbitrary photo collections. At any time, users can define a region of interest either explicitly or implicitly, and the application retrieves, scores, groups and shows a matching subset of the photos. Users can then select a photo to project it onto the 3D model, to inspect the photo separately, or to teleport to the position the photo was taken from. Unlike previous approaches for joint 2D-3D model exploration, our interface has been specifically adapted to VR. We conducted a user study and found that the application greatly facilitates navigation and provides a fast, intuitive access to the available photos. The application supports any modern browser running on desktop, mobile and VR headset systems.The digitization of the St. Quirze de Pedret models was partially funded by the Spanish Ministry of Economy and Competitiveness and FEDER Grants TIN2017-88515-C2-1-R, the Romanesque Pyrenees, Space of Artistic Confluences II (PRECA II) project (HAR2017-84451-P, Universitat de Barcelona) and the JPICH0127 EU project Enhancement of Heritage Experiences: the Middle Ages; Digital Layered Models of Architecture and Mural Paintings over Time (EHEM).Peer ReviewedPostprint (published version