Extended reality and informative models for the architectural heritage: from scan-to-BIM process to virtual and augmented reality

[EN] The dissemination of the tangible and intangible values of heritage building represents one of the most important objectives in the field of Digital Cultural Heritage (DCH). In recent years, different studies and research applied to heritage monuments have shown how it is possible to improve the awareness of the architectural heritage through the integration of latest developments in the field of 3D survey, 3D modelling, Building Information Modeling (BIM) and eXtended Reality (XR). On the other hand, this digital workflow requires a huge amount of data sources and a holistic approach to reach a high level of information sharing coming from different disciplines and sectors such as restoration, geomatics, 3D virtual museums and serious gaming. In conjunction with entertainment software and gaming, this research shows the main results obtained during the generative process of digital environments oriented to improve the level of information and to enrich the contents coming from the informative models. The case study is represented by one of the most important Lombard monuments: the Basilica of Sant’Ambrogio in Milan. This study, starting from the 3D survey and the data collection of the historical records of the church, improves the creation of an XR experience that reaches a new level of interactivity for different types of devices (desktop, mobile, VR headset) and users (experts, non-experts).Highlights:Generative modelling requirements and novel grades of generations (GOG) and accuracy (GOA) are presented in order to improve the digitisation of built heritage from the 3D survey, reducing time and costs of the scan-to-BIM process.The holistic value of generative modelling allows experts to create digital worlds able to faithfully and accurately represent the detected reality and improve new immersive environments for Virtual Reality (VR) and Augmented Reality (AR) projects.Immersive environments are created with a mixture of the latest generation software and hardware, allowing users to discover the hidden historical values of built heritage with new levels of interactivity and information.[ES] La divulgación de los valores tangibles e intangibles en el patrimonio construido representan uno de los objetivos más importantes en el campo del Patrimonio Cultural Digital (DCH). En los últimos años, diferentes estudios han demostrado la posibilidad de mejorar el conocimiento de nuestro patrimonio construido a través de la integración de los últimos desarrollos en el campo del levantamiento topográfico 3D, modelado 3D, Modelado de Información de la Construcción (BIM) y Realidad eXtendida (RX). Del mismo modo, se ha comprobado que este flujo de trabajo requiere de una gran cantidad de datos y un enfoque holístico con el fin de alcanzar un nivel alto de información compartida entre las diferentes disciplinas y sectores involucrados, como por ejemplo, la restauración, la geomática, los museos virtuales 3D y la industria del videojuego. Junto con el software de entretenimiento y los videojuegos, esta investigación muestra los principales resultados obtenidos durante el proceso de generación de entornos digitales orientados a mejorar el nivel de información y el contenido de uno de los monumentos más importantes en Lombardía: la Basílica de Sant’Ambrogio en Milán. Este estudio, que se inicia con el levantamiento 3D y la toma de datos provenientes de los registros históricos de la iglesia, está orientado a mejorar la creación de una experiencia de realidad extendida, con el objetivo de alcanzar varios niveles de interactividad a través de diferentes dispositivos (escritorio, móvil, auriculares de realidad virtual) y usuarios (expertos, no-expertos).Banfi, F.; Brumana, R.; Stanga, C. (2019). Realidad extendida y modelos informativos en patrimonio arquitectónico: del proceso scan-to-BIM a la realidad virtual y aumentada. Virtual Archaeology Review. 10(21):14-30. https://doi.org/10.4995/var.2019.1192314301021Attar, R., Hailemariam, E., Glueck, M., Tessier, A., McCrae, J., & Khan, A. (2010). BIM-based building performance monitor. Invited Video at Symposium on Simulation for Architecture and Urban Design (SimAUD) 2010. Orlando, FL, USA. Retrieved from https://www.autodeskresearch.com/publications/bimdashboardvideoAzhar, S. (2011). Building Information Modeling (BIM): Trends, benefits, risks, and challenges for the AEC industry. Leadership and management in engineering, 11(3), 241-252. https://doi.org/10.1061/(ASCE)LM.1943-5630.0000127Banfi, F. (2016). Building Information Modelling-A novel parametric modeling approach based on 3D surveys of historic architecture. In Euro-Mediterranean Conference (pp. 116-127). Cham: Springer. https://doi.org/10.1007/978-3-319-48496-9_10Banfi, F. (2017). BIM orientation: grades of generation and information for different type of analysis and management process. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(2/W5), 57-64. 10.5194/isprs-archives-XLII-2-W5-57-2017Banfi, F. (2019). Holistic generative modeling process for HBIM (Doctoral dissertation, Politecnico di Milano, Italy).Banfi, F., Chow, L., Ortiz, M. R., Ouimet, C., & Fai, S. (2018). Building Information Modeling for cultural heritage: The management of generative process for complex historical buildings. In M. Ioannides, E. Fink, R. Brumana, P. Patias, A. Doulamis, J. Martins, M. Wallace (Eds.), Digital Cultural Heritage (pp. 119-130). Cham: Springer. https://doi.org/10.1007/978-3-319-75826-8_10Barazzetti, L., Banfi, F., Brumana, R., Gusmeroli, G., Previtali, M., & Schiantarelli, G. (2015). Cloud-to-BIM-to-FEM: Structural simulation with accurate historic BIM from laser scans. Simulation Modelling Practice and Theory, 57, 71-87. https://doi.org/10.1016/j.simpat.2015.06.004Blanco-Pons, S., Carrión-Ruiz, B., Lerma, J. L., & Villaverde, V. (2019). Design and implementation of an augmented reality application for rock art visualization in Cova dels Cavalls (Spain). Journal of Cultural Heritage. https://doi.org/10.1016/j.culher.2019.03.014Bonavita, A. (2004). Pellegrino Tibaldi a Milano: i lavori alla cupola e al coro della basilica di Sant'Ambrogio. Arte Lombarda,140(1), 89-91.Bradley, A., Li, H., Lark, R., & Dunn, S. (2016). BIM for infrastructure: An overall review and constructor perspective. Automation in Construction, 71, 139-152. https://doi.org/10.1016/j.autcon.2016.08.019Brumana, R., Condoleo, P., Grimoldi, A., & Previtali, M. (2019). Towards a semantic based hub platform of vaulted systems: HBIM Meets A GeoDB. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(2/W11), 301-308. https://doi.org/10.5194/isprs-archives-XLII-2-W11-301-2019Brumana, R., Condoleo, P., Grimoldi, A., Banfi, F., Landi, A. G., Previtali, M., (2018a). HR LOD based HBIM to detect influences on geometry and shape by stereotomic construction techniques of brick vaults. Applied Geomatics, 10(4), 529-543. https://doi.org/10.1007/s12518-018-0209-3Brumana, R., Condoleo, P., Grimoldi, A., Landi, A. G., Attico, D., Turrina, A., & Banfi, F. (2018b). HBIM Feeding Open Access Vault Inventory Through GeoDB HUB. In M. Ioannidess, E. Fink, R. Brumana, P. Patias, A. Doulamis, J. Martins, M. Wallace (Eds.), Digital Heritage (pp. 27-38). Cham: Springer. https://doi.org/10.1007/978-3-030-01762-0_3Brumana, R., Della Torre, S., Previtali, M., Barazzetti, L., Cantini, L., Oreni, D., & Banfi, F. (2018c). Generative HBIM modelling to embody complexity (LOD, LOG, LOA, LOI): surveying, preservation, site intervention-the Basilica di Collemaggio (L'Aquila). Applied Geomatics, 10(4), 545-567. https://doi.org/10.1007/s12518-018-0233-3Cabrelles, M., Blanco-Pons, S., Carrión-Ruiz, B., & Lerma, J. L. (2018). From multispectral 3D recording and documentation to development of mobile apps for dissemination of cultural heritage. In T. Levy, I. Jones (Eds.), Cyber-Archaeology and Grand Narratives (pp. 67-90). Cham: Springer. https://doi.org/10.1007/978-3-319-65693-9_5Cassina, F. (1844). Le fabbriche più cospicue di Milano. Milan: Domenico Pedrinelli (book consulted at the Historical Library, Politecnico di Milano).Castellazzi, G., D'Altri, A., Bitelli, G., Selvaggi, I., & Lambertini, A. (2015). From laser scanning to finite element analysis of complex buildings by using a semi-automatic procedure. Sensors, 15(8), 18360-18380. https://doi.org/10.3390/s150818360De Dartein, F. (1865-82). Etude sur l'architecture lombarde et sur les origines de l'architecture romano-bizantine. Paris: Dunod (book consulted at the Historical Library, Politecnico di Milano).Degli Abbati, S., D'Altri, A. M., Ottonelli, D., Castellazzi, G., Cattari, S., de Miranda, S., & Lagomarsino, S. (2019). Seismic assessment of interacting structural units in complex historic masonry constructions by nonlinear static analyses. Computers & Structures, 213, 51-71. https://doi.org/10.1016/j.compstruc.2018.12.001Gatti Perrer, M. L. (1995). Il tempio ininterrotto. Milan: Vita e Pensiero.Georgopoulos, A., (2018). Contemporary Digital Technologies at the Service of Cultural Heritage. In B. Chanda, S. Chaudhuri, S. Chaudhury (Eds.), Heritage Preservation (pp. 1-20). Singapore: Springer. https://doi.org/10.1007/978-981-10-7221-5_1Gusmeroli, G., & Schiantarelli, G., (2014). From laser clouds to BIM and finite element analysis: the case study of Castel Masegra (Master thesis, Politecnico di Milano, Italy).Ioannides, M., Magnenat-Thalmann, N., & Papagiannakis, G. (Eds.) (2017). Mixed Reality and Gamification for Cultural Heritage. Cham: Springer. https://doi.org/10.1007/978-3-319-49607-8Korumaz, M., Betti, M., Conti, A., Tucci, G., Bartoli, G., Bonora, V., Güleç Korumaz, A., & Fiorini, L. (2017). An integrated Terrestrial Laser Scanner (TLS), Deviation Analysis (DA) and Finite Element (FE) approach for health assessment of historical structures. A minaret case study. Engineering Structures, 153(15), 224-238. https://doi.org/10.1016/j.engstruct.2017.10.026Kumar, S. S., & Cheng, J. C. (2015). A BIM-based automated site layout planning framework for congested construction sites. Automation in Construction, 59, 24-37. https://doi.org/10.1016/j.autcon.2015.07.008Lai, Z., Hu, Y. C., Cui, Y., Sun, L., & Dai, N. (2017). Furion: Engineering high-quality immersive virtual reality on today's mobile devices. Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking (pp. 409-421). Snowbird, Utah, USA. https://doi.org/10.1145/3117811.3117815Landriani, G. (1889). La basilica ambrosiana fino alla sua trasformazione in chiesa lombarda a volte. Milan: Hoepli.Lerma, J. L., Navarro, S., Cabrelles, M., & Villaverde, V. (2010). Terrestrial laser scanning and close range photogrammetry for 3D archaeological documentation: the Upper Palaeolithic Cave of Parpalló as a case study. Journal of Archaeological Science, 37(3), 499-507. https://doi.org/10.1016/j.jas.2009.10.011Lusuardi Siena, S. (1997). Ambrogio, il costruttore sapiente. In M. Rizzi, C. Pasini, M. P. Rossignani (Eds.), La città e la sua memoria: Milano e la tradizione di Sant'Ambrogio (pp. 34-35). Milan: Electa.Mazzetti, P., Latre, M. Á., Ernst, J., Brumana, R., Brauman, S., Nativi, S. ( 2015). Virtual hubs for facilitating access to open data. EGU General Assembly Conference held 12-17 April, 2015 in Vienna, Austria.Obidah, R., & Bein, D. (2019). Game based learning using unreal engine. In S. Latifi (Eds.), 16th International Conference on Information Technology-New Generations (pp. 513-519). Cham: Springer. https://doi.org/10.1007/978-3-030-14070-0_72Oreni, D., Della Torre, S., Brumana, R., & Banfi, F. (2017). Disegno e modellazione parametrica per la conservazione di un edificio monumentale danneggiato da un evento sismico. Dal rilievo al cantiere. ANANKE Speciale Geores 2017, 153-158.Oreni, D., Brumana, R., Della Torre, S., Banfi, F., & Previtali, M. (2014). Survey turned into HBIM: the restoration and the work involved concerning the Basilica di Collemaggio after the earthquake (L'Aquila). ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(5), 267-273. https://doi.org/10.5194/isprsannals-II-5-267-2014Patetta, L. (2001). Bramante e la sua cerchia a Milano e in Lombardia 1480-1500. Milan: Skira.Peroni, A. (1987). Tradizione e innovazione in S. Ambrogio romanico. In C. Bertelli (Eds.), Il Millennio Ambrosiano, (pp. 156-172). Milan: Electa.Piegl, L., & Tiller, W. (2012). The NURBS book. Springer Science & Business Media. Cham: Springer.Previtali, M., Barazzetti, L., Banfi, F., & Roncoroni, F. (2019). Informative content models for infrastructure load testing management: the Azzone Visconti Bridge in Lecco. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(2/W11), 995-1001. https://doi.org/10.5194/isprs-archives-XLII-2-W11-995-2019Reggiori, F. (1949). Superstantia Sancti Abrosii Mediolani. Milan: A. Faccioli.Rossi, F. M. (1884). Cronaca dei ristauri e delle scoperte fatte nell'insigne basilica di S. Ambrogio dall'anno 1857 al 1876. Milan: Tipografia di S. Giuseppe.Santana Quintero, M., Blake, B., Eppich, R., & Ouimet, C. (2008). Heritage documentation for conservation: partnership in learning. In 16th ICOMOS General Assembly and International Symposium: 'Finding the spirit of place - between the tangible and the intangible'. Quebec, Canada.Stanga, C., Spinelli, C., Brumana, R., Oreni, D., Valente, R., Banfi, F. (2017). A N-D virtual notebook about the Basilica of S. Ambrogio in Milan: information modeling for the communication of historical phases subtraction process. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42(2/W5), 653-660. https://doi.org/10.5194/isprs-archives-XLII-2-W5-653-2017Statham, N. (2019). Scientific rigour of online platforms for 3D visualization of heritage. Virtual Archaeology Review, 10(20), 1-16. https://doi.org/10.4995/var.2019.9715Summa, A. (1995). La scultura medioevale nella basilica di S. Ambrogio. In M. L. Gatti Perrer (Eds.), La basilica di S. Ambrogio: il tempio ininterrotto. Milan: Vita e pensiero.Tucci, G., & Lerma, J. L. (2018). Special Issue GEORES2017. Geomatics and restoration: Conservation of cultural heritage in the Digital Era. Preface. Applied Geomatics, 10(4), 277-278. https://doi.org/10.1007/s12518-018-0247-xVolk, R., Stengel, J., & Schultmann, F. (2014). Building Information Modeling (BIM) for existing buildings - Literature review and future needs. Automation in construction, 38, 109-127. https://doi.org/10.1016/j.autcon.2013.10.023Wetzel, E. M., & Thabet, W. Y. (2015). The use of a BIM-based framework to support safe facility management processes.Automation in Construction, 60, 12-24. https://doi.org/10.1016/j.autcon.2015.09.00